We do not currently know of any antonyms for on the plus side.
The prepositional phrase on the plus side is defined as: