At the fishing pier in Semporna town, fishermen who have just returned from a fishing trip are still trapped by life with knitted brows.