Why Your AI Bill Will Double Before It Gets Better
About this episode
The episode discusses the rising costs of running AI at scale and strategies for cost optimization with insights from Josh Collier.
In this episode, we're joined by Josh Collier , FinOps Lead at Superhuman (formerly Grammarly), to explore what it really costs to run AI at scale and why the rules of the game changed faster than anyone expected. We discuss how AI token costs dropped 80% in two years, why that trend has sharply reversed with frontier models doubling in price, and how Josh rebuilt a single LLM workflow that cost $400k a month down to $80k by rethinking the architecture. He also shares how a cost calculator built in 15 minutes transformed the way his team estimates spend before running experiments, and why research-led optimization is the only kind that works without degrading the product. Along the way, we cover hidden costs most teams miss, the trade-off between Azure reserved capacity and OpenAI Priority Processing, why fixed subscription pricing is broken in an AI-native world, vendor lock-in risk, and what OpenAI's Guaranteed Capacity announcement really signals about where vendor relationships are heading next. Superhuman: https://superhuman.com Josh Collier: https://www.linkedin.com/in/josh-collier-945b7029/ Demetrios: https://www.linkedin.com/in/dpbrinkm Timestamps: [00:00] OpenAI…
More episodes of Agentic Conversations (formally mlops.community)
- The Winchester Mystery House Problem in AI Development
- How Predictive Analytics Stops Budget Overruns Before They Happen?
- How To Delegate To An Agent Like You Would An Employee?
- MCP Goes Stateless
- AI Hype vs. Real Value
Explore listener stats, chart rankings, contacts and more on the Agentic Conversations (formally mlops.community) podcast page.