“Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations” by Subhash Kantamneni, kitft, Euan Ong, Sam Marks

About this episode

The episode discusses Natural Language Autoencoders and their application in generating explanations for LLM activations and model auditing.

Abstract We introduce Natural Language Autoencoders (NLAs), an unsupervised method for generating natural language explanations of LLM activations. An NLA consists of two LLM modules: an activation verbalizer (AV) that maps an activation to a text description and an activation reconstructor (AR) that maps the description back to an activation. We jointly train the AV and AR with reinforcement learning to reconstruct residual stream activations. Although we optimize for activation reconstruction, the resulting NLA explanations read as plausible interpretations of model internals that, according to our quantitative evaluations, grow more informative over training. We apply NLAs to model auditing. During our pre-deployment audit of Claude Opus 4.6, NLAs helped diagnose safety-relevant behaviors and surfaced unverbalized evaluation awareness—cases where Claude believed, but did not say, that it was being evaluated. We present these audit findings as case studies and corroborate them using independent methods. On an automated auditing benchmark requiring end-to-end investigation of an intentionally-misaligned model, NLA-equipped agents outperform baselines and can succeed even without…

People in this episode

Guests: Subhash Kantamneni, kitft, Euan Ong, Sam Marks

Topics covered

• Natural Language Processing
• Machine Learning
• Model Interpretability
• Unsupervised Learning
• AI Safety

Keywords

• Natural Language Autoencoders
• LLM activations
• activation verbalizer
• activation reconstructor
• model auditing

Mentioned in this episode

Organizations: Claude Opus

More episodes of LessWrong (30+ Karma)

• “The Uncertainty That Matters Isn’t Fundamental” by jimmy · June 13, 2026 · 22 min
• [Linkpost] “US government directive to suspend access to Fable 5 and Mythos 5” by Capybasilisk · June 13, 2026 · 0 min
• “Claude Fable 5 and Mythos 5: The System Card” by Zvi · June 12, 2026 · 60 min
• “Citations Needed: Magic Encyclopedias to Save the World” by Oliver Sourbut · June 12, 2026 · 10 min
• “Simulating Simulators” by kromem · June 12, 2026 · 27 min
• “Implications of Continual Learning for LLM Agents: Introduction” by RohanS, Rauno Arike, Owen Terry, Achu Menon, Zhijing Jin, Francis Rhys Ward, Seth Herd · June 12, 2026 · 14 min