The Bitter Lesson

The Bitter Lesson

From Linear Digressions by Katie Malone

March 15, 2026 · 19 min

About this episode

The episode discusses the concept of the Bitter Lesson in AI development, emphasizing the importance of scale over sophistication in model building.

Every AI builder knows the anxiety: you spend months engineering prompts, tuning pipelines, and chaining calls together — then a new model drops and half your work evaporates overnight. It turns out researchers have been wrestling with this exact dynamic for 30 years, and they keep arriving at the same uncomfortable answer. That answer is called the Bitter Lesson — and understanding it might be the most important thing you can do for whatever you're building right now. From Deep Blue to AlexNet to modern LLMs, scale keeps beating sophistication, and knowing which side of that line your work falls on makes all the difference. Links - Richard Sutton, "The Bitter Lesson" - Alon Halevy, Peter Norvig, and Fernando Pereira, "The Unreasonable Effectiveness of Data" - Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, "ImageNet Classification with Deep Convolutional Neural Networks"

People in this episode

Host: Katie Malone

Topics covered

  • AI development
  • machine learning
  • Bitter Lesson
  • model evolution
  • data effectiveness
  • scale vs sophistication

Keywords

  • AI
  • Bitter Lesson
  • machine learning
  • Deep Blue
  • AlexNet
  • LLMs
  • data
  • model tuning

Mentioned in this episode

Books & works: The Bitter Lesson, ImageNet Classification with Deep Convolutional Neural Networks

More episodes of Linear Digressions

Explore listener stats, chart rankings, contacts and more on the Linear Digressions podcast page.