About this episode

Hugo Bowne-Anderson interviews Sebastian Raschka about the future of LLM architecture and the importance of understanding AI models.

If you take a model release as an anchor point , let’s say Nemotron 3 or Qwen 3.5, you can go in both directions : You can either plug them into an agent and play around with that, or you can look, okay, what does the model look like under the hood ? What are the ingredients? What type of attention mechanism do they use? What are currently research techniques that could make that even better in the next generation of models? What can we swap out, basically? And I’m interested in both of these! Sebastian Raschka , Independent AI Researcher and author of Build a Large Language Model from Scratch , joins Hugo to talk about what’s changed in AI architecture, from post-training to hybrid models, and why understanding what’s under the hood matters more than ever for developers building in the agentic era . Sebastian’s upcoming book, Build a Reasoning Model from Scratch , currently available for pre-order on Amazon and in early access on Manning ! Vanishing Gradients is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. We Discuss: * Ed Tech for Agents : should we design educational content specifically for agentic…

People in this episode

Host: Hugo Bowne-Anderson

Guest: Sebastian Raschka

Topics covered

• AI architecture
• large language models
• agentic systems
• hybrid models
• inference scaling
• educational content

Keywords

• AI architecture
• large language models
• inference scaling
• hybrid architectures
• agentic systems

Mentioned in this episode

Products: Build a Large Language Model from Scratch, Build a Reasoning Model from Scratch, Nemotron 3, Qwen 3.5

More episodes of Vanishing Gradients

• Next Level AI Evals for 2026 · April 23, 2026 · 54 min
• Privacy Theater Is Not Privacy Engineering: What It Actually Takes to Ship Safe AI · April 15, 2026 · 1h 7m
• Episode 72: Why Agents Solve the Wrong Problem (and What Data Scientists Do Instead) · March 20, 2026 · 1h 34m
• Episode 71: Durable Agents - How to Build AI Systems That Survive a Crash with Samuel Colvin · February 17, 2026 · 51 min
• Episode 70: 1,400 Production AI Deployments · February 11, 2026 · 1h 10m
• Episode 69: Python is Dead. Long Live Python! With the Creators of pandas & Parquet · February 2, 2026 · 55 min

Explore listener stats, chart rankings, contacts and more on the Vanishing Gradients podcast page.