We do not currently know of any antonyms for pride yourself on.
For similar words to pride yourself on, try: