About this episode

The episode discusses the evolution of AI evaluations and their importance in product development, featuring insights from experts in the field.

There are a lot of reasons why we should do AI evals. For many companies doing AI evals is the way to build the feedback loop into the product development lifecycle. So it is like your compass. We’re using AI evals as a compass to guide product development and also product iteration. And also, many times we need evals to function as the pass or fail gate in release decisions. Whether this product is good enough for release or whether it is good enough for experiment, evals are also used in that. Stella Wenxing Liu , Head of Applied Science at ASU , and Eddie Landesberg , Staff Data Scientist at Google , join Hugo to talk about why AI evaluation is evolving from “vibe checks” into a rigorous, multi-disciplinary science and how causal inference will take AI evals to the next level in 2026 . Vanishing Gradients is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. They Discuss: * Team-Centric AI Evals , integrating product managers , data scientists , and SMEs under a “benevolent dictator” (or not!) to ensure comprehensive and effective evaluation; * Custom Evaluation Metrics , moving beyond generic vendor metrics…

People in this episode

Host: Hugo Bowne-Anderson

Guests: Stella Wenxing Liu, Eddie Landesberg

Topics covered

• AI evaluation
• product development
• causal inference
• custom evaluation metrics
• team-centric evaluations
• policy evaluation

Keywords

• AI evals
• product iteration
• feedback loop
• evaluation metrics
• causal inference
• AB testing
• data science

Mentioned in this episode

Organizations: ASU, Google

More episodes of Vanishing Gradients

• Privacy Theater Is Not Privacy Engineering: What It Actually Takes to Ship Safe AI · April 15, 2026 · 1h 7m
• LLM Architecture in 2026: What You Need to Know with Sebastian Raschka · April 12, 2026 · 1h 18m
• Episode 72: Why Agents Solve the Wrong Problem (and What Data Scientists Do Instead) · March 20, 2026 · 1h 34m
• Episode 71: Durable Agents - How to Build AI Systems That Survive a Crash with Samuel Colvin · February 17, 2026 · 51 min
• Episode 70: 1,400 Production AI Deployments · February 11, 2026 · 1h 10m
• Episode 69: Python is Dead. Long Live Python! With the Creators of pandas & Parquet · February 2, 2026 · 55 min

Explore listener stats, chart rankings, contacts and more on the Vanishing Gradients podcast page.