About this episode

Sebastian Raschka discusses the advancements and challenges in LLMs as we head into 2026.

Sebastian Raschka joins the MAD Podcast for a deep, educational tour of what actually changed in LLMs in 2025 — and what matters heading into 2026. We start with the big architecture question: are transformers still the winning design, and what should we make of world models, small “recursive” reasoning models and text diffusion approaches? Then we get into the real story of the last 12 months: post-training and reasoning. Sebastian breaks down RLVR (reinforcement learning with verifiable rewards) and GRPO, why they pair so well, what makes them cheaper to scale than classic RLHF, and how they “unlock” reasoning already latent in base models. We also cover why “benchmaxxing” is warping evaluation, why Sebastian increasingly trusts real usage over benchmark scores, and why inference-time scaling and tool use may be the underappreciated drivers of progress. Finally, we zoom out: where moats live now (hint: private data), why more large companies may train models in-house, and why continual learning is still so hard. If you want the 2025–2026 LLM landscape explained like a masterclass — this is it. Sources: The State Of LLMs 2025: Progress, Problems, and Predictions…

People in this episode

Host: Matt Turck

Guest: Sebastian Raschka

Topics covered

• LLMs
• reinforcement learning
• architecture
• reasoning
• inference scaling
• evaluation

Keywords

• LLMs
• RLVR
• GRPO
• transformers
• reasoning
• inference scaling
• benchmaxxing
• private data
• continual learning

Mentioned in this episode

Organizations: Sebastian Raschka Website, Blog, LinkedIn

More episodes of The MAD Podcast with Matt Turck

• OpenAI Board Member Zico Kolter on the Real Risks of Frontier AI · May 7, 2026 · 1h 17m
• AI is Already Building AI | Google DeepMind’s Mostafa Dehghani · April 2, 2026 · 1h 5m
• Benedict Evans: OpenAI’s Moat Problem & the Future of Software · March 19, 2026 · 1h 1m
• Everything Gets Rebuilt: The New AI Agent Stack | Harrison Chase, LangChain · March 12, 2026 · 47 min
• AI That Can Prove It’s Right: Verification as the Missing Layer in AI — Carina Hong · February 26, 2026 · 1h 4m
• Voice AI’s Big Moment: Why Everything Is Changing Now (ft. Neil Zeghidour, Gradium AI) · February 19, 2026 · 1h 23m

Explore listener stats, chart rankings, contacts and more on the The MAD Podcast with Matt Turck podcast page.