We do not currently know of any adverbs for leaving.
Using available adjectives, one could potentially construct nonstandard adverbs such as leavelessly, leavingly or leftly.
Note, however, that these words will not generally be contained within standard English language lexicons.