The word fisherman refers a person who fishes recreationally or professionally.
There are no categorical antonyms for this word.
The word fishermen is the plural of fisherman.