About this episode

The episode discusses how to evaluate generative AI through various metrics and challenges.

We unpack how to evaluate AI that writes and creates, not just predicts. Why perplexity captures surprise, why a low perplexity score isn’t a guarantee of correctness, and how precision, recall, and the harmonic F1 balance model performance. We compare BLEU and ROUGE, explore Retrieval-Augmented Generation to stay faithful to private data, and discuss out-of-domain challenges, agentic AI, and the guardrails shaping the future. Note: This podcast was AI-generated, and sometimes AI can m...

Mentioned in this episode

Organizations: AI, BLEU

More episodes of Intellectually Curious

• 835 Pages to 100% Rust: Inside the AI Swarm That Rebuilt SQLite · July 21, 2026 · 7 min
• Replit's Self-Driving Company: How AI Agents Turn Engineers into Directors · July 20, 2026 · 5 min
• Kimi K3 Unleashed: The Open-Source AI that Multiplies Human Insight · July 19, 2026 · 6 min
• Neurosymbolic Sportscasting: Real-Time AI Narrates RoboCup · July 18, 2026 · 6 min
• Synchronizing Nano-Oscillators for Next-Generation AI Computing Hardware · July 17, 2026 · 5 min
• AI Disproves the Benjamini–Hochberg Conjecture · July 16, 2026 · 6 min

Explore listener stats, chart rankings, contacts and more on the Intellectually Curious podcast page.