Researchers Map Attention Circuits in 70B Models, Revealing Interpretable Reasoning Paths

A team from MIT and Stanford has published a landmark study mapping 80% of the attention circuits responsible for reasoning behaviors in a 70B parameter language model. The work reveals consistent, human-readable patterns for how models internally represent factual chains. The findings have direct implications for AI safety and the development of more reliable alignment methods.