We do not currently know of any antonyms for give one's attention to.
For similar words to give one's attention to, try: