The Race to Production-Grade Diffusion LLMs with Stefano Ermon - #764

About this episode

Stefano Ermon discusses the adaptation of diffusion models for language generation and their advantages over traditional LLMs.

Today, we're joined by Stefano Ermon, associate professor at Stanford University and CEO of Inception Labs to discuss diffusion language models. We dig into how diffusion approaches—traditionally used for images—are being adapted for text and code generation, the technical challenges of applying continuous methods to discrete token spaces, and how diffusion models compare to traditional autoregressive LLMs. Stefano introduces Mercury 2, a commercial-scale diffusion LLM that can generate multiple tokens simultaneously and achieve inference speeds 5-10x faster than small frontier models, paving the way for latency-sensitive applications like voice interactions and fast agentic loops. We also cover the open research challenges in diffusion LLM training, serving infrastructure requirements, and post-training for diffusion-based systems. Finally, Stefano shares his perspective on whether diffusion models can rival or surpass autoregressive LLMs at scale, the advantages for highly controllable generation, and what the future of multimodal diffusion models might look like. The complete show notes for this episode can be found at https://twimlai.com/go/764.

People in this episode

Host: Sam Charrington

Guest: Stefano Ermon

Topics covered

• diffusion language models
• text generation
• code generation
• technical challenges
• inference speeds
• multimodal diffusion models

Keywords

• diffusion models
• language models
• inference speeds
• text generation
• code generation
• Mercury 2
• autoregressive LLMs

Mentioned in this episode

Organizations: Stanford University, Inception Labs

Products: Mercury 2

More episodes of The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

• Is RAG Dead? Lessons from Building AI for Tax Law with Alex Bowcut - #769 · June 9, 2026 · 52 min
• Relational Foundation Models for Enterprise Data with Jure Leskovec - #768 · May 21, 2026 · 1h 6m
• How to Find the Agent Failures Your Evals Miss with Scott Clark - #767 · May 7, 2026 · 53 min
• How to Engineer AI Inference Systems with Philip Kiely - #766 · April 30, 2026 · 55 min
• How Capital One Delivers Multi-Agent Systems with Rashmi Shetty - #765 · April 16, 2026 · 54 min
• Agent Swarms and Knowledge Graphs for Autonomous Software Development with Siddhant Pardeshi - #763 · March 10, 2026 · 1h 16m