The past tense of put by for a rainy day is also put by for a rainy day.
The third-person singular simple present indicative form of put by for a rainy day is puts by for a rainy day.
The present participle of put by for a rainy day is putting by for a rainy day.
The past participle of put by for a rainy day is put by for a rainy day.