“You Are Not Immune To Mode Collapse” by J Bostock

About this episode

J Bostock discusses the phenomenon of mode collapse in AI and its implications across various fields.

“Mode collapse” is a few things. First it was an observation about how early image generating AIs often collapsed to producing just the modal output from their training distribution (something very common, like a house with a white picket fence and a tree in the garden). Then it was the observation that this effect seemed to occur extremely quickly when AIs were trained on AI-generated inputs. After that, it became the copium du jour of AI-is-hitting-a-wall folks for a while, who thought that the AI industry would ouroboros itself out of existence (and that there was, therefore, no need to confront any of the issues that smarter than human AIs might bring up). And then it was forgotten, because it turns out you can train on AI-generated inputs just fine, if you know what you’re doing. It's also the reason why grant-making organisations have such strong inertia, why all of your favourite band's songs sound the same after the third album, and why you should specialise even if there are no gains from trade. The Image Generator Imagine an image generating AI, which gets something like this as input: Original image: https://commons.wikimedia.org/wiki/File:Dog_Breeds.jpg And suppose…

People in this episode

Host: LessWrong

Guest: J Bostock

Topics covered

• AI
• mode collapse
• image generation
• grant-making
• music
• specialization

Keywords

• mode collapse
• AI
• image generation
• grant-making
• music
• specialization

Mentioned in this episode

Organizations: LessWrong

Books & works: You Are Not Immune To Mode Collapse

More episodes of LessWrong (30+ Karma)

• “The Uncertainty That Matters Isn’t Fundamental” by jimmy · June 13, 2026 · 22 min
• [Linkpost] “US government directive to suspend access to Fable 5 and Mythos 5” by Capybasilisk · June 13, 2026 · 0 min
• “Claude Fable 5 and Mythos 5: The System Card” by Zvi · June 12, 2026 · 60 min
• “Citations Needed: Magic Encyclopedias to Save the World” by Oliver Sourbut · June 12, 2026 · 10 min
• “Simulating Simulators” by kromem · June 12, 2026 · 27 min
• “Implications of Continual Learning for LLM Agents: Introduction” by RohanS, Rauno Arike, Owen Terry, Achu Menon, Zhijing Jin, Francis Rhys Ward, Seth Herd · June 12, 2026 · 14 min