Voice Agent Use Cases
From Agentic Conversations (formally mlops.community) by Demetrios
May 1, 2026 · 51 min
About this episode
This episode explores the engineering and design challenges of building production-grade voice AI systems at scale.
This episode is brought to you by the MLflow team. Check out more information at MLflow.org . What does it actually take to build voice AI at a billion-interaction scale? This episode features an ex-Amazon voice AI engineer who built customer support systems handling 2 billion+ interactions — now working on next-gen voice agent platforms. Anurag digs deep into the real engineering tradeoffs, design patterns, and use cases that separate production-grade voice agents from demos. Voice Agent Use Cases // MLOps Podcast #372 with Anurag Beniwal, Member of the Technical Staff at ElevenLabs 🎙️ Topics covered: 🔹 Cascaded vs. speech-to-speech — Why cascaded systems still win in production, and how to make them feel natural without sacrificing control 🔹 Latency masking — Foreground/background model architecture and how to buy yourself time while deep retrieval runs 🔹 Constellation of models — Using Haiku for tool calling, fine-tuned smaller models for response generation, and why "one model for everything" breaks at scale 🔹 Turn-taking & ASR challenges — Why voice is harder than chat: accents, noise, silence detection, and domain-specific fine-tuning 🔹 Level 1 vs Level…
People in this episode
Host: Demetrios
Guest: Anurag Beniwal
Topics covered
- voice AI
- engineering tradeoffs
- design patterns
- customer support systems
- voice agent platforms
- inbound sales agents
- level 1 vs level 2 support
Keywords
- voice agents
- customer support
- cascaded systems
- latency masking
- ASR challenges
- sales agents
- reservations
Sponsors
Hyperbolic, MLflow
Mentioned in this episode
Organizations: MLflow, Amazon, ElevenLabs
More episodes of Agentic Conversations (formally mlops.community)
- The Winchester Mystery House Problem in AI Development · August 24, 2026 · 60 min
- How Predictive Analytics Stops Budget Overruns Before They Happen? · August 20, 2026 · 29 min
- How To Delegate To An Agent Like You Would An Employee? · August 17, 2026 · 55 min
- Why Your AI Bill Will Double Before It Gets Better · August 3, 2026 · 32 min
- MCP Goes Stateless · July 27, 2026 · 52 min
- AI Hype vs. Real Value · July 24, 2026 · 43 min
Explore listener stats, chart rankings, contacts and more on the Agentic Conversations (formally mlops.community) podcast page.