The noun epsilon can be countable or uncountable.

In more general, commonly used, contexts, the plural form will be epsilons.

However, in more specific contexts, the plural form can also be epsila e.g. in reference to various types of epsila or a collection of epsila.