How I AI

I’m tired of new models. Every week there’s a new benchmark, a new frontier intelligence claim, a new thing to test. But here we are, because Opus 5 just dropped and I’ve had real hands-on time with it, so you’re getting the honest version.This is my full Opus 5 review: personality analysis, live benchmark results from my 7-model How I AI eval, and an actual verdict on whether I’m swapping it in. Spoiler: the answer surprised me.What you’ll learn:Why I think we’ve hit an intelligence overhang and what that means for which model variables actually matter nowHow Opus 5’s “neurotic” personality showed up in real coding sessions, including a merge conflict it refused to touchWhat I learned from asking both Opus 5 and GPT‑5.6 Sol “who’s smarter, you or me?”Where Opus 5, GPT‑5.6 Sol, Sonnet 5, and Gemini 3.1 Pro actually landed on the HIA benchmark leaderboardThe one use case where Opus 5 earned straight 5s from meMy actual plan for using Opus 5 going forward—In this episode, I cover:(00:00) Opus 5 is here(03:15) First impressions(06:12) Opus 5 vs. GPT‑5.6 Sol personality comparison(14:39) Claude Slop: the verbosity problem and why it makes my blood boil(16:55) How the How I AI benchmark works (7 models, 6 tasks, blind scoring)(18:30) Live benchmark results: the leaderboard reveal(23:25) My verdict and how I’ll actually use Opus 5—Tools referenced:• Claude Opus 5:• Anthropic blog: https://www.anthropic.com/news• GPT‑5.6 Sol: https://openai.com/index/previewing-gpt-5-6-sol/• Sonnet 5: https://www.anthropic.com/news/claude-sonnet-5• Gemini 3.1 Pro: https://deepmind.google/models/gemini/pro/—Where to find Claire Vo:ChatPRD: https://www.chatprd.ai/Website: https://clairevo.com/LinkedIn: https://www.linkedin.com/in/clairevo/X: https://x.com/clairevo—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

In this episode, I sit down with Ankur Goyal, founder and CEO of Braintrust, the AI evals and observability platform used by teams like Notion, Stripe, Vercel, and Zapier. This one is for the senior engineers, staff engineers, VPs of engineering, and CTOs in my audience. We get into how coding agents can take on deeply technical architecture and infrastructure work that no single human engineer could tackle before, and then we demystify evals so you can use them to make your AI products better without touching the implementation.What you’ll learn:How Ankur uses Codex to run week-long benchmark experiments across database indexes, column store formats, and execution engines to speed up slow queriesWhy he argues there’s no excuse to skip rigorous benchmarking now that agents can run them tirelesslyThe “agent line” framework: how to decide which decisions, directions, and interactions you can hand off to an agentHow I think about the practical vs. theoretical quality of AI on hard technical problems, and why human attention decays on tedious workWhy evals are the modern version of a PRD, and how to encode “what good looks like” so a model can figure out the “how”How to build a scoring function live and let an agent improve your prompt inside a safe playgroundHow Ankur turned his designer David’s taste into a repeatable eval so quality scales beyond one personWhy fixing your CI is the highest-leverage way to speed up engineering velocity—Brought to you by:Guru—The AI layer of truthPersona—Trusted identity verification for any use case—In this episode, we cover:(00:00) Introduction to Ankur Goyal(03:00) Using AI agents for database optimization(06:10) Running exhaustive benchmarks with coding agents(09:03) Why staff engineers are wrong about AI limitations(11:30) The “agent line” framework for delegation(14:00) Ankur’s workflow: running 4 to 6 concurrent agents(17:16) Technical setup: foreground agents, background agents, and cloud environments(20:32) Spending time with AI tools(23:06) Demystifying evals(26:02) Live demo: Building an eval for documentation answers(30:20) The alternative to evals: vibe checks and whack-a-mole(32:09) Capturing designer taste in scoring functions(33:13) Quick recap(33:44) Managing velocity and throughput(35:40) Why CI/CD investment is critical for AI-accelerated teams(37:30) Ankur’s prompting strategy when agents fail(39:10) Closing thoughts and how to connect—Tools referenced:• Braintrust: https://www.braintrust.dev/• Codex: https://openai.com/codex/• GPT 5.4: https://developers.openai.com/api/docs/models/gpt-5.4• Claude: https://claude.ai/—Other references:• GPT 5.5 just did what no other model could: https://www.lennysnewsletter.com/p/gpt-55-just-did-what-no-other-model• Paul Graham’s Maker vs. Manager Schedule: http://www.paulgraham.com/makersschedule.html• tmux: https://github.com/tmux/tmux• Chris Tate at Vercel: https://www.linkedin.com/in/ctatedev/—Where to find Ankur Goyal:LinkedIn: https://www.linkedin.com/in/ankrgyl/—Where to find Claire Vo:ChatPRD: https://www.chatprd.ai/Website: https://clairevo.com/LinkedIn: https://www.linkedin.com/in/clairevo/X: https://x.com/clairevo—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.