EVA - A Framework for Evaluating Voice Agents by ServiceNow
From ServiceNow Insights by ServiceNow Community
April 29, 2026 · 30 min
About this episode
The episode discusses the challenges of evaluating voice AI agents and introduces ServiceNow's open-source framework for establishing industry standards.
Voice AI agent evaluation — why it's fundamentally harder than text, how cascade failures derail conversations invisibly, and ServiceNow's open-source framework to establish industry evaluation standards. Featuring real audio examples showing authentication failures, leaked reasoning, and latency problems. WHAT WE COVER TARA BOGAVELLI — Research Engineer, ServiceNow Leading the open-source voice agent evaluation framework. Explains why existing benchmarks don't measure what matters and what ServiceNow is releasing to establish industry standards. KATRINA STANKIEWICZ — Staff Machine Learning Engineer, ServiceNow Cascade model architecture expert. Breaks down STT → LLM → TTS failure modes, named entity transcription challenges, and real audio example analysis. GABRIELLE GAUTHIER MELANÇON — Staff Applied Research Scientist, ServiceNow Multi-language evaluation specialist. Reveals why Large Audio Language Models lag behind, the native speaker requirement, and bot-to-bot simulation methodology. CHAPTERS 0:00 Introduction — The evaluation gap 1:11 ServiceNow's Open-Source Framework Announcement — Tara Bogavelli 2:43 Meet…
People in this episode
Guests: Tara Bogavelli, Katrina Stankiewicz, Gabrielle Gauthier Melançon
Topics covered
- Voice AI evaluation
- Open-source frameworks
- Cascade failures
- Machine learning challenges
- Audio language models
- Evaluation metrics
Keywords
- Voice AI
- evaluation framework
- cascade failures
- machine learning
- audio language models
- evaluation metrics
Mentioned in this episode
Organizations: ServiceNow
More episodes of ServiceNow Insights
- Being AI Native at ServiceNow · May 27, 2026 · 24 min
- Day One Ready: What New Engineers Need to Know About AI — Engineering Now Unlocked · May 13, 2026 · 30 min
- AI Control Tower - Governing AI at Scale with ServiceNow · April 15, 2026 · 19 min
- AGENTIC AI - The Future of Work and the Agents Building It · April 2, 2026 · 23 min
- The Hidden Cost of How Your Business Actually Works | Process Mining with Dan Grady · March 18, 2026 · 24 min
- The Human in the Loop | Ethical AI with Di Le · March 4, 2026 · 29 min
Explore listener stats, chart rankings, contacts and more on the ServiceNow Insights podcast page.