Development of Linear Truth Encodings in Language Models: A Replication Study

A growing body of interpretability work reports that language models represent the truth of a statement along a single linear direction. This project asks how reliably that finding reproduces, and how such a direction comes to exist during training.

This is a short note on work that is forthcoming and currently under review, so I will keep it high level.1A preprint and details will appear once available; in the meantime see my Google Scholar profile. The starting point is a claim that has drawn a lot of attention in interpretability: that a language model encodes whether a statement is true or false in a roughly linear way, so that a single direction in its representation space separates true statements from false ones. If that is right, it is a striking piece of structure, and it matters for how we probe and steer models.

Our study is a replication. Rather than propose a new method, we set out to test whether the reported linear truth directions hold up when the analysis is rerun carefully, and to look at where they come from. The developmental angle is the part I care most about: instead of examining only a finished model, we ask how a linear encoding of truth emerges over the course of training. Tracking a probe direction as a model trains tells us whether such structure appears early or late, whether it is stable or drifts, and how much it depends on the specific choices an experimenter makes.

Replication studies are undervalued in this area. Interpretability results often rest on particular datasets, probe designs, and model checkpoints, and small changes can move the conclusions. By re-examining the linear truth-encoding finding and its development, we hope to say something concrete about how robust it is and under what conditions it holds. I will update this page with results and a link once the paper is public.