BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data

A multilingual companion to the developmentally plausible pretraining agenda: instead of one child's language environment in English, a curated approximation of it across dozens of languages.

The BabyLM effort has spent the last few years asking a pointed question: how much of what large language models learn actually needs internet-scale data, and how much could a model learn from something closer to the amount of language a child hears while acquiring their first tongue? BabyBabelLM1Jumelet, Fourtassi, Haga et al. BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data, 2026. takes that question out of its heavily English setting and asks it in many languages at once. I am a co-author on this work.

What the resource is

BabyBabelLM is a collection of pretraining datasets meant to approximate the language a child is exposed to, from birth through the years of native acquisition. Rather than scraping whatever text is abundant, the team prioritised the kinds of input a child actually encounters: child-directed speech, educational materials, children's books, and transcribed conversation. Synthetic data was deliberately left out, which keeps the corpus honest as a model of real developmental input.

Coverage spans 45 languages, organised into three tiers by how much data could be gathered. The top tier reaches roughly 100 million English-word-equivalents, the middle tier around 10 million, and the smallest tier around 1 million. Sorting languages into tiers rather than forcing a single fixed budget is a practical concession: language communities differ enormously in how much child-appropriate text exists, and pretending otherwise would either starve the well-resourced languages or fabricate data for the rest.

How it was built

The collection was led language by language. Researchers who know a given language ran its collection effort, drawing on language-specific sources and supplementing with multilingual resources where needed. Everything then went through uniform preprocessing and standardised metadata annotation, so the corpora are comparable across languages rather than a patchwork of incompatible formats. The release also carries document-level licensing information and source attribution, which matters if the resource is going to be reused responsibly.

What ships with it

Beyond the raw curated data, hosted on HuggingFace, BabyBabelLM provides an evaluation suite for each language and a set of baseline models, both monolingual and multilingual. That combination is what turns a data dump into a benchmark: you can train under a controlled, developmentally motivated budget and then measure against a shared yardstick within and across languages.

The reason I care about this is that it makes multilingual developmental questions answerable on comparable footing. If we want to ask whether smaller, plausibly scaled data teaches models the right things, we should be able to ask it in Japanese or Basque as readily as in English. BabyBabelLM is an attempt to lay down that common ground.