“Notes on Transformer Consciousness” by slavachalnev

About this episode

The episode explores the speculative nature of conscious experience in transformers and compares the processes of decoding and pre-filling in their operation.

Assuming transformers can have conscious experience, what would that experience be like? Transformers[1] are a structured grid of layers and token positions and we can use this structure to reason about their internal experience. Epistemic status: very speculative. I've ordered this writeup approximately by how much I've thought it through and how much I believe it. Decode vs Prefill I claim that the experience of decode is identical to the experience of prefill. To see why this is the case, picture a transformer that's generating a token at a time and zoom in on a single layer at the latest token position. There are two major components: the MLP, which processes this position's residual stream, and the attention block, which can "see" all the previous positions at this layer. So in the diagram below, each position in the transformer only has access to what happened in the lower left box. Now let's take the trace we just computed and pass it into the transformer as prefill. The activations will, of course, all be the same as during decode, but the way they are computed will also be the same. From the perspective of the MLP and the attention block [...] --- Outline: (00:32) Decode…

People in this episode

Guest: slavachalnev

Topics covered

• transformer consciousness
• artificial intelligence
• epistemology
• neural networks
• speculative reasoning

Keywords

• transformers
• conscious experience
• decode
• prefill
• MLP
• attention block
• epistemic status

Mentioned in this episode

Books & works: Notes on Transformer Consciousness

More episodes of LessWrong (30+ Karma)

• “The Uncertainty That Matters Isn’t Fundamental” by jimmy · June 13, 2026 · 22 min
• [Linkpost] “US government directive to suspend access to Fable 5 and Mythos 5” by Capybasilisk · June 13, 2026 · 0 min
• “Claude Fable 5 and Mythos 5: The System Card” by Zvi · June 12, 2026 · 60 min
• “Citations Needed: Magic Encyclopedias to Save the World” by Oliver Sourbut · June 12, 2026 · 10 min
• “Simulating Simulators” by kromem · June 12, 2026 · 27 min
• “Implications of Continual Learning for LLM Agents: Introduction” by RohanS, Rauno Arike, Owen Terry, Achu Menon, Zhijing Jin, Francis Rhys Ward, Seth Herd · June 12, 2026 · 14 min