About this episode
This episode explores the engineering and design challenges of building production-grade voice AI systems at scale.
This episode is brought to you by the MLflow team. Check out more information at MLflow.org . What does it actually take to build voice AI at a billion-interaction scale? This episode features an ex-Amazon voice AI engineer who built customer support systems handling 2 billion+ interactions — now working on next-gen voice agent platforms. Anurag digs deep into the real engineering tradeoffs, design patterns, and use cases that separate production-grade voice agents from demos. Voice Agent Use Cases // MLOps Podcast #372 with Anurag Beniwal, Member of the Technical Staff at ElevenLabs 🎙️ Topics covered: 🔹 Cascaded vs. speech-to-speech — Why cascaded systems still win in production, and how to make them feel natural without sacrificing control 🔹 Latency masking — Foreground/background model architecture and how to buy yourself time while deep retrieval runs 🔹 Constellation of models — Using Haiku for tool calling, fine-tuned smaller models for response generation, and why "one model for everything" breaks at scale 🔹 Turn-taking & ASR challenges — Why voice is harder than chat: accents, noise, silence detection, and domain-specific fine-tuning 🔹 Level 1 vs Level…
People in this episode
Host: Demetrios
Guest: Anurag Beniwal
Topics covered
- voice AI
- engineering tradeoffs
- design patterns
- customer support systems
- voice agent platforms
- inbound sales agents
- level 1 vs level 2 support
Keywords
- voice agents
- customer support
- cascaded systems
- latency masking
- ASR challenges
- sales agents
- reservations
Sponsors
Hyperbolic, MLflow
Mentioned in this episode
Organizations: MLflow, Amazon, ElevenLabs
More episodes of MLOps.community
- MCP, Agents & the $40M Bet on Multiplayer AI · June 12, 2026 · 1h 21m
- From Single-Player to Multi-Player: Operating AI Agents at Scale · June 9, 2026 · 56 min
- The Control-vs-Magic Spectrum Building Agents · June 5, 2026 · 43 min
- Logs Are All You Need: Rethinking Observability with AI Agents · June 2, 2026 · 47 min
- AI Is Fast. AI Projects Are Slow. Let's Fix That. · May 29, 2026 · 57 min
- Architecting Modern AI Systems: Platforms, Agents, and Integration · May 28, 2026 · 57 min
Explore listener stats, chart rankings, contacts and more on the MLOps.community podcast page.