GPT 5.4 is a big step for Codex
About this episode
Nathan Lambert reviews GPT 5.4 and discusses its performance in agent tasks compared to traditional benchmarks.
I’m a little late to this model review, but that has given me more time to think about the axes that matter for agents. Traditional benchmarks reduce model performance to a single score of correctness – they always have because that was simple, easy to quickly use to gauge performance, and so on. This is also advice that I give to people trying to build great benchmarks – it needs to reduce to one number that is interpretable. This is likely still going to be true in a year or two, and benchmarks for agents will be better, but for the time being it doesn’t really map to what we feel because agentic tasks are all about a mix of correctness, ease of use, speed, and cost. Eventually benchmarks will individually address these. Where GPT 5.4 feels like another incremental model on some on-paper benchmarks, in practice it feels like a meaningful step in all four of those traits. GPT 5.4 in Codex, always on fast mode and high or extra-high effort, is the first OpenAI agent that feels like it can do a lot of random things you can throw at it. Interconnects AI is a reader-supported publication. Consider becoming a subscriber. I haven’t been particularly deep in software engineering over…
People in this episode
Host: Nathan Lambert
Topics covered
- AI
- model review
- benchmarks
- agent performance
- software engineering
Keywords
- GPT 5.4
- Codex
- benchmarks
- agent performance
- software engineering
- AI
Mentioned in this episode
Organizations: OpenAI
Products: GPT 5.4, Codex
More episodes of Interconnects
- My bets on open models, mid-2026
- The inevitable need for an open model consortium
- Claude Mythos and misguided open-weight fearmongering
- Gemma 4 and what makes an open model succeed
- Lossy self-improvement
- What comes next with open models
Explore listener stats, chart rankings, contacts and more on the Interconnects podcast page.