About this episode

Edwin Chen discusses the challenges and nuances of AI evaluation and the divergent paths of frontier labs in developing AGI.

Edwin Chen is the founder and CEO of Surge AI, the data infrastructure company behind nearly every major frontier model. Surge works with OpenAI, Anthropic, Meta, and Google, providing the high-quality data and evaluation infrastructure that powers their models. Edwin reveals why optimizing for popular benchmarks like LMArena is "basically optimizing for clickbait," how one frontier lab's models regressed for 6-12 months without anyone knowing, and why the industry's approach to measurement is fundamentally broken. Jacob and Edwin discuss what actually makes elite AI evaluators, why "there's never going to be a one size fits all solution" for AI models, and how frontier labs are taking surprisingly divergent paths to AGI.

People in this episode

Host: Jacob Effron

Guest: Edwin Chen

Topics covered

• AI evaluation
• data infrastructure
• frontier models
• AGI development
• benchmarking
• machine learning

Keywords

• Surge AI
• frontier labs
• AI models
• LMArena
• data quality
• evaluation infrastructure
• machine learning benchmarks

Mentioned in this episode

Organizations: Surge AI, OpenAI, Anthropic, Meta, Google

More episodes of Unsupervised Learning with Jacob Effron

• Ep 85: Has AI Infra Stabilized, FM Vibe Shift, & What's Next for Coding Agents · April 23, 2026 · 55 min
• Ep 84: OpenAI’s Chief Scientist on Continual Learning Hype, RL Beyond Code, & Future Alignment Directions · April 9, 2026 · 59 min
• Ep 83: Owning the System of Record, AI-Native Org Charts, & Why ITSM is The Most Vulnerable Legacy Category · April 2, 2026 · 54 min
• Ep 82: Behind Legora's $550M Raise, Model Competition, Doubling Revenue Every Quarter, & US Expansion · March 11, 2026 · 54 min
• Ep 81: Ex-OpenAI Researcher On Why He Left, His Honest AGI Timeline, & The Limits of Scaling RL · January 29, 2026 · 1h 3m
• AI Vibe Check: The Actual Bottleneck In Research, SSI’s Mystique, & Spicy 2026 Predictions · December 18, 2025 · 1h 18m

Explore listener stats, chart rankings, contacts and more on the Unsupervised Learning with Jacob Effron podcast page.