AI Inference Costs Are Crushing SaaS Gross Margins — Here's What to Do About It

About this episode

Ben Murray discusses the impact of AI inference costs on SaaS gross margins and offers strategies for managing these expenses.

Is your AI SaaS company skating on thin ice because of exploding compute costs you're not tracking? In episode #365, Ben Murray tackles one of the most pressing financial challenges facing AI-first SaaS companies: the structural margin compression caused by LLM inference costs. Traditional SaaS was built on near-zero marginal cost per customer — that era is over. If you're building on top of AI, every prompt, query, and agentic workflow is a hard COGS line that scales with revenue, and if you're not managing it, it will quietly destroy your unit economics. Why AI-first SaaS companies are running 50–60% gross margins (vs. 70–80% for legacy SaaS) — and what Bessemer data shows about AI supernovas with margins as low as 25%. How inference and compute costs differ fundamentally from traditional SaaS COGS — and why they won't scale down the way hosting costs did Why token costs vary wildly (from $1–2 per million to $30–180+ for frontier models) and how that variability makes feature-level economics a CFO priority 5 tactical ways to reduce LLM spend: model routing, prompt caching, context compaction, semantic caching, and batch processing How to set up your GL accounts and COGS…

People in this episode

Host: Ben Murray

Topics covered

• AI SaaS
• inference costs
• gross margins
• unit economics
• LLM spend
• COGS tracking

Keywords

• AI inference costs
• SaaS gross margins
• LLM
• COGS
• unit economics
• compute costs
• feature-level economics

Mentioned in this episode

Organizations: Bessemer

More episodes of SaaS Metrics School

• 12 Steps to Creating an Outcome-based Pricing Plan · June 12, 2026 · 7 min
• 5 Takeaways for CFOs from the 2026 AI Pricing Report · June 10, 2026 · 7 min
• Your AI Subscription Pricing Is Losing Money on the Customers You Care About Most · June 2, 2026 · 5 min
• 4 SaaS P&L Metrics That Break When You Kill Per-Seat Pricing · May 31, 2026 · 5 min
• Per-Seat Pricing Is Dying: What the Shift to Usage-Based SaaS Means for Your Margins · May 29, 2026 · 5 min
• The Two SaaStr Annual Slides Every SaaS Operator Needs to See Today · May 20, 2026 · 3 min