The past tense of lay by is laid by.
The third-person singular simple present indicative form of lay by is lays by.
The present participle of lay by is laying by.
The past participle of lay by is laid by.