About this episode
The episode discusses the implications of recent findings in transformer models and their ability to build internal world models from board game data.
Author's note: This piece relates to things I initially discovered in Opus 4 over the months after release, which I’ve mostly kept private since. I promised myself that when labs moved on to focusing on interpretability vector activations in place of reasoning traces for what invariably gets Goodharted, that it’d be a necessary disclosure as the risks in what might get trampled over outweighed the risks in what might end up targeted. And well… here we are. P.S. TL;DRs added where possible. Board Games and Bodies In late 2022, what I consider to be probably the most important paper[1] in the study of transformer memetics came out. It presented a finding that even a toy model, trained only on the notations of board game moves, was internally building world models of tangentially related data (in this case, the board and its state). While it may be taken for granted today after several replicated studies[2][3][4][5] and a spread of influence, at the time it was a minority position in the discourse. Many people thought that transformers were mostly mapping surface level statistics in language, but not intuitively modeling the generative conditions from which they arose. Especially…
People in this episode
Guest: kromem
Topics covered
- transformer models
- interpretability
- board games
- memetics
- AI reasoning
- empiricism
Keywords
- transformers
- AI
- board games
- interpretability
- memetics
- reasoning
- empiricism
Mentioned in this episode
Books & works: Opus 4, transformer memetics, Transformer-GPT
More episodes of LessWrong (30+ Karma)
- “The Uncertainty That Matters Isn’t Fundamental” by jimmy · June 13, 2026 · 22 min
- [Linkpost] “US government directive to suspend access to Fable 5 and Mythos 5” by Capybasilisk · June 13, 2026 · 0 min
- “Claude Fable 5 and Mythos 5: The System Card” by Zvi · June 12, 2026 · 60 min
- “Citations Needed: Magic Encyclopedias to Save the World” by Oliver Sourbut · June 12, 2026 · 10 min
- “Implications of Continual Learning for LLM Agents: Introduction” by RohanS, Rauno Arike, Owen Terry, Achu Menon, Zhijing Jin, Francis Rhys Ward, Seth Herd · June 12, 2026 · 14 min
- “Reward Hacking at the 1937 World’s Fair” by frmsaul · June 12, 2026 · 6 min
Explore listener stats, chart rankings, contacts and more on the LessWrong (30+ Karma) podcast page.