The noun cellular glass insulation can be countable or uncountable.

In more general, commonly used, contexts, the plural form will also be cellular glass insulation.

However, in more specific contexts, the plural form can also be cellular glass insulations e.g. in reference to various types of cellular glass insulations or a collection of cellular glass insulations.