We do not currently know of any antonyms for having a rapport with.
For similar words to having a rapport with, try: