Ep 255: Does this research explain how LLMs work?

About this episode

The episode explores research papers related to how LLMs work, focusing on the Bayesian Attention Trilogy and includes an interview with one of the authors.

I take a look at these three papers: 1. https://www.arxiv.org/abs/2512.22471 2. https://arxiv.org/abs/2512.23752 3. https://arxiv.org/abs/2512.22473 Collectively titled "The Bayesian Attention Trilogy" along with some other material - in particular an interview with one of the authors "Vishal Misra" - https://www.engineering.columbia.edu/faculty-staff/directory/vishal-misra For those familiar with my output on this you can probably skip to about halfway through at 42:40. Prior to this is a lot of background on Induction, Bayesianism, Critical Rationalism and so on that people may have heard from me before in different contexts - although for what it's worth these are new ways of expressing those ideas. At the end I am reacting to a video found here: https://www.youtube.com/watch?v=uRuY0ozEm3Q

People in this episode

Guest: Vishal Misra

Topics covered

• LLMs
• Bayesianism
• Induction
• Critical Rationalism

Keywords

• research
• papers
• interview
• video reaction

Mentioned in this episode

Products: Bayesian Attention Trilogy

Books & works: The Bayesian Attention Trilogy

More episodes of ToKCast

• Ep 262: David Deutsch's "The Fabric of Reality" Chapter 13 "The Four Strands" Part 3 · May 4, 2026 · 1h 4m
• Ep 261: Objective Morality II · April 23, 2026 · 39 min
• Ep 260: I don't believe it! · April 5, 2026 · 1h 7m
• Ep 259: Reaction to Michael Levin Part 3: Bait and Switch, Podcastistan and Surprising(?) algorithms. · March 11, 2026 · 60 min
• Ep 258: Reaction to Michael Levin, Part 2: Bioelectricity, Consciousness and Knowing · February 16, 2026 · 1h 39m
• Ep 257: Reaction to Michael Levin, Part 1: Categories and Continuums. · February 9, 2026 · 1h 8m