Evidence of Emergent World Models Found in Transformers Trained Purely on Text

A new paper from Princeton and UC Berkeley presents evidence that large language models trained exclusively on text develop internal world representations encoding spatial and causal relationships beyond what surface-level pattern matching would predict. Researchers used a novel probing methodology to extract 3D spatial maps from model activations showing consistent object permanence and causal inference properties. The findings add empirical weight to the hypothesis that scale alone may be sufficient for grounded world model formation.