About this episode

Noam Brown discusses the limitations of traditional AI benchmarks and the impact of large-scale test-time compute on model evaluation and safety.

When a new AI model drops, it’s judged based on a static benchmark grid that doesn’t account for how long the model is allowed to think. How then should we measure a model’s true capability? OpenAI research scientist Noam Brown returns to talk with Sarah Guo about his latest essay on why the AI industry’s traditional benchmark grids are broken, and how large-scale test-time compute is fundamentally changing how models are evaluated. Noam explains how, if properly scaffolded, today’s models can reason for weeks or even months on complex tasks. He also discusses real-world implications of test-time compute, from building poker solver bots to disproving legendary math conjectures. Together, they also unpack the large gaps in current AI safety frameworks, explore the bottlenecks for recursive self-improvement, and look ahead at the future of multi-agent collaboration and global knowledge sharing. Read more: Implications of Large-Scale Test-Time Compute Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @polynoamial | @OpenAI Chapters: 00:00 – Cold Open 00:43 – Noam Brown Introduction 01:23 – Why…

People in this episode

Host: Sarah Guo

Guest: Noam Brown

Topics covered

• AI benchmarks
• test-time compute
• AI safety
• recursive self-improvement
• multi-agent collaboration

Mentioned in this episode

Organizations: OpenAI

More episodes of No Priors: Artificial Intelligence | Technology | Startups

• Travel Through the Lens of AI with with Booking.com CEO Glenn Fogel · July 9, 2026 · 41 min
• How Nuclear Will Unlock Energy Abundance with Valar Atomics Founder Isaiah Taylor · July 2, 2026 · 1h 1m
• Re-engineering the Semiconductor Supply Chain with Intel CEO Lip-Bu Tan · June 18, 2026 · 45 min

Explore listener stats, chart rankings, contacts and more on the No Priors: Artificial Intelligence | Technology | Startups podcast page.