Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?

Forthcoming work asking a simple but loaded question: if you train language models on different languages entirely separately, do their internal representations still end up speaking to each other? Currently under review.

This paper is under review, so I will sketch the question rather than report any results.1Under review at EMNLP 2026 (Main Conference); no public page yet. See my Google Scholar profile for updates. The standard route to cross-lingual ability is joint training: pour many languages into one model so that shared parameters are forced to find common ground. That works, but it makes it hard to tell where the alignment comes from. Is a shared multilingual space something the model discovered about language, or something we manufactured by mixing everything together and giving it no choice?

The question

To separate those two possibilities, this work looks at monolingual models trained independently, each on a single language, with no shared vocabulary, no parallel data, and no joint objective tying them together. The question is whether their representation spaces nonetheless line up, in the sense that a learned mapping can carry meaning from one model's space into another's. If independently trained models turn out to be compatible after the fact, that points at something the models are converging on rather than something training forced them to share.

The title frames this as convergence on universal representations. The interesting outcome is not a foregone conclusion in either direction. It is quite possible that models trained apart end up too idiosyncratic to reconcile, in which case cross-lingual structure really does depend on joint training. It is also possible that they converge to broadly compatible geometries, which would say something about how much shared structure language itself imposes.

I find this appealing because it treats universality as a hypothesis to be tested rather than an assumption. It also connects to my broader interest in what models learn under controlled, plausible training conditions rather than at maximum scale with everything mixed in. I will replace this placeholder with a proper write-up, including the actual method and findings, once the paper is public.