1025: Word Gravity: How Transformers Bend Space, with Dr. Luis Serrano
About this episode
Dr. Luis Serrano discusses the concept of word gravity in transformer architectures and its implications for machine learning.
In Episode #1025, Dr. Luis Serrano (Founder of Serrano Academy) joins Jon Krohn to explain the paper he co-authored on the curved spacetime of transformer architectures, in which attention stops being a lookup table and becomes something closer to gravity: words bend the space around them, and the embedding of "bank" visibly curves toward "river" as it travels through the layers of the network. In this episode, he recreates Eddington’s 1919 eclipse experiment inside a transformer, draws the line between an LLM workflow and an actual agent, explains why agent evaluation is a step harder than evaluating an essay, and gives the cleanest account of GRPO you will hear. Additional materials: https://www.superdatascience.com/1025 Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information. In this episode you will learn: (00:10:53) What changed, and what survived, between the two editions of Grokking Machine Learning (00:27:48) Word gravity: how attention pulls "bank" toward "river" (00:42:12) Why RAG is an LLM…
More episodes of Super Data Science: ML & AI Podcast with Jon Krohn
- 1024: In Case You Missed It in August 2026
- 1023: Agentic AI Skills That Matter Now, with Aishwarya Srinivasan
- 1022: CLAUDE.md, AGENTS.md, Skills, Hooks and Subagents: A Field Guide to Steering AI Agents
- 1021: How dbt Won Analytics Engineering, with dbt Lab’s CEO Tristan Handy
- 1020: How to Choose Model Size and Effort Level: The Two Critical Dials
- 1019: Anyone Can Write Code Now, So What Gets You Hired? (With Priyanka Vergadia)
Explore listener stats, chart rankings, contacts and more on the Super Data Science: ML & AI Podcast with Jon Krohn podcast page.