Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction

Child–caregiver dialogue is contingent: turns are prompt, direct, and meaningful. We ask whether a small model trained on 100M words can be taught to hold up its end of that kind of exchange.

When a caregiver and a child talk, the back-and-forth has a property developmental linguists call contingency. A response arrives quickly, it connects to what the other person just said, and it moves the exchange somewhere.1Salhan, Gu, Rooein, Galvan-Sosa, Gaudeau, Caines, Yuan & Buttery. Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction, BabyLM Workshop, EMNLP 2025. That is easy to describe and hard to build. In this paper, with my co-authors, I take contingency seriously as a target for a BabyLM: a language model constrained to roughly the amount of language a child hears, on the order of 100 million words.

The framing borrows from Vygotsky. A learner has a zone of proximal development, the space of things they cannot yet do alone but can manage with the right support. A caregiver, or a teacher, supplies that support through well-timed demonstrations. We ask what the analogue of a teacher demonstration is for a small dialogue model, and whether providing one at the right moment helps the model produce more contingent turns.

ContingentChat

To study this concretely we built ContingentChat, a framework for benchmarking and then improving multi-turn contingency in a data-limited model. The benchmark side gives us a way to measure whether a response is actually grammatical and coherent in context, rather than merely fluent in isolation. The intervention side is a post-training step: we align the base BabyLM against a dataset assembled for this purpose, so that the model's replies become more grammatical and more cohesive across turns.

The alignment dataset is the load-bearing piece. Rather than hoping contingency emerges from more pretraining, we curate examples that show what a good contingent turn looks like and train the model against them. This is where the teacher-demonstration idea earns its keep: the demonstrations are the supervision signal that pulls the model's dialogue behaviour into a better region.

What the decoding could and could not do

We also looked at inference-time control. The intuition was that adaptive decoding, adjusting how the model samples depending on the conversational state, might supply some of the same benefit more cheaply than retraining. In practice the adaptive strategies we tried added little on top of the post-training. The gains came from the alignment data, not from clever sampling. I find that a useful negative result: it says the problem is one of what the model has been taught to prefer, not merely how we read out its distribution at test time.

Taken together, the results position contingency as a genuinely difficult optimisation goal for small models. Post-training on targeted demonstrations moves grammaticality and coherence in the right direction, and the improvements are real, but holding a contingent conversation remains a demanding ask of a model that has seen so little language. The paper received an Outstanding Paper Award at the BabyLM Workshop,2First BabyLM Workshop, EMNLP 2025, Suzhou, China. See the ACL Anthology entry. which I read as encouragement to keep pushing on interaction, and not only on the static benchmarks that dominate the sample-efficient pretraining literature.