About this episode

The episode discusses the concept of the pre-training wall in AI and its implications for future developments in machine learning.

I've been confusing Don with frontier-lab links late at night for a bit. Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't growing. The frontier labs call this the pre-training wall. A leaked Google memo from 2023 argued they had no moat. R1 is on GitHub. Llama is on Hugging Face. OpenAI's secondary-market valuation has climbed past $850 billion. Don was confused. So he came over and we made an episode about it. Episode Page Support The Show Subscribe To The Podcast Join The Newsletter

Explore listener stats, chart rankings, contacts and more on the CoRecursive: Coding Stories podcast page.