About this episode
Nathan Lambert reviews GPT 5.4 and discusses its performance in agent tasks compared to traditional benchmarks.
I’m a little late to this model review, but that has given me more time to think about the axes that matter for agents. Traditional benchmarks reduce model performance to a single score of correctness – they always have because that was simple, easy to quickly use to gauge performance, and so on. This is also advice that I give to people trying to build great benchmarks – it needs to reduce to one number that is interpretable. This is likely still going to be true in a year or two, and benchmarks for agents will be better, but for the time being it doesn’t really map to what we feel because agentic tasks are all about a mix of correctness, ease of use, speed, and cost. Eventually benchmarks will individually address these. Where GPT 5.4 feels like another incremental model on some on-paper benchmarks, in practice it feels like a meaningful step in all four of those traits. GPT 5.4 in Codex, always on fast mode and high or extra-high effort, is the first OpenAI agent that feels like it can do a lot of random things you can throw at it. Interconnects AI is a reader-supported publication. Consider becoming a subscriber. I haven’t been particularly deep in software engineering over…
People in this episode
Host: Nathan Lambert
Topics covered
- AI
- model review
- benchmarks
- agent performance
- software engineering
Keywords
- GPT 5.4
- Codex
- benchmarks
- agent performance
- software engineering
- AI
Mentioned in this episode
Organizations: OpenAI
Products: GPT 5.4, Codex
More episodes of Interconnects
- Claude Fable 5 and new AI safety fables · June 9, 2026 · 12 min
- Farewell Ai2 · June 2, 2026 · 16 min
- Open and closed models are on different exponentials · June 1, 2026 · 7 min
- Some ideas for what comes next, May 2026 · May 26, 2026 · 10 min
- Notes from inside China's AI labs · May 7, 2026 · 17 min
- The distillation panic · May 4, 2026 · 9 min
Explore listener stats, chart rankings, contacts and more on the Interconnects podcast page.