The noun mathematical markup language can be countable or uncountable.
In more general, commonly used, contexts, the plural form will also be mathematical markup language.
However, in more specific contexts, the plural form can also be mathematical markup languages e.g. in reference to various types of mathematical markup languages or a collection of mathematical markup languages.