The past tense of fall away is fell away.
The third-person singular simple present indicative form of fall away is falls away.
The present participle of fall away is falling away.
The past participle of fall away is fallen away.