We do not currently know of any antonyms for had an affair with.
For similar words to had an affair with, try: