E21: The Evolution of AI PMs, Prototyping, Evals, and Real Impact with Aman Khan

About this episode

The episode explores the evolution of AI product management with insights from Aman Khan on prototyping and evaluations.

In this episode of Practically Intelligent, we sit down with Aman Khan, Director of Product at Arize AI, to unpack what “AI product management” really means. We dig into how tools like Cursor and Claude Code are changing the way PMs prototype, making disposable, working prototypes a core part of the craft again. From there, we go deep on evals and the three common workflows teams fall into, how they go wrong, and what it looks like to build evals that actually tie to business metrics instead of just vibes and thumbs-ups. Finally, Aman shares his own practical rituals like building golden datasets, running weekly data reviews, and doing product tear-downs, plus tips any PM can use to build real AI product sense and get ready for an agentic future.

People in this episode

Hosts: Sinan Ozdemir, Akshay Bhushan

Guest: Aman Khan

Topics covered

• AI product management
• prototyping
• evaluations
• business metrics
• data reviews
• product sense

Keywords

• AI
• product management
• prototyping
• evaluations
• business metrics
• data reviews
• golden datasets

Mentioned in this episode

Organizations: Arize AI

Products: Cursor, Claude Code

More episodes of Practically Intelligent

• E20 – From Classrooms to Curiosity: AI & The Future of Learning with Katelyn Donnelly · June 4, 2025 · 42 min
• E19 – From Pixels to Prompts: How AI Is Reshaping UX Design · April 10, 2025 · 48 min
• E18: From Negative One to Zero: The Early Startup Journey with Arian Agrawal · March 3, 2025 · 42 min
• E17: Graph Databases Are the Future of RAG and AI with Sudhir Hasbe · November 26, 2024 · 51 min
• E16: AI in Action: Regulating the Future with Professor Philip Treleaven · October 10, 2024 · 40 min
• OH1: Office Hours are Open! · August 29, 2024 · 13 min