"The paper that killed deep learning theory" by LawrenceC

About this episode

The episode discusses a pivotal paper that challenged classical deep learning theory and its implications on the field.

Around 10 years ago, a paper came out that arguably killed classical deep learning theory: Zhang et al. 's aptly titled Understanding deep learning requires rethinking generalization. Of course, this is a bit of an exaggeration. No single paper ever kills a field of research on its own, and deep learning theory was not exactly the most productive and healthy field at the time this was published. But if I had to point to a single paper that shattered the feeling of optimism at the time, it ...

People in this episode

Guest: LawrenceC

Topics covered

• deep learning
• theory
• generalization
• research
• optimism

Keywords

• deep learning
• theory
• generalization
• Zhang et al.
• research paper

Mentioned in this episode

Books & works: Understanding deep learning requires rethinking generalization

More episodes of LessWrong (Curated & Popular)

• "PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett · June 12, 2026 · 2 min
• "Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models" by Anders Cairns Woodruff, Francis Rhys Ward, Dewi Gould, Rauno Arike, Jason R Brown, Jo Jiao, wlanderson, ariana_azarbal, harrymayne, Patrick Leask · June 11, 2026 · 10 min
• "Even “illegible” Mythos reasoning traces seem pretty legible" by faul_sname · June 11, 2026 · 8 min
• "Sequent: scale and automation for higher confidence in alignment" by Geoffrey Irving, Alex HT, Jesse Hoogland, Daniel Murfet, Jacob Pfau, Marco Cozzi, Stan van Wingerden · June 10, 2026 · 23 min
• "The Machines Lack Honour" by Raymond Douglas · June 10, 2026 · 20 min
• "My favorite depiction of utopia" by Caleb Biddulph · June 4, 2026 · 58 min