AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart
About this episode
Matthieu Wyart discusses how deep networks can discover abstractions that shallow models miss and the implications for machine learning.
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstWhy can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality.The conversation moves from jamming transitions and rough loss surfaces to Chomsky, context-free grammars and machine creativity. Wyart explains why next-token prediction can still recover compositional structure, where current systems fall short of genuine scientific invention, and why predicting latent representations rather than raw tokens could make learning far more sample-efficient.They also examine diffusion models, neural scaling laws and the limits of physics-inspired theory. The final question is on a personal note: if mistakes are the price of leaving the beaten path, how much scientific risk is worth taking?---TIMESTAMPS:00:00:00 Can machines learn abstractions from data?00:02:00 Notion agentic workspace00:02:49 From…
People in this episode
Host: Tim Scarfe
Guest: Matthieu Wyart
Topics covered
- deep learning
- abstraction
- machine creativity
- neural networks
- statistical physics
- dimensionality reduction
Keywords
- deep networks
- abstractions
- curse of dimensionality
- machine learning
- latent representations
- diffusion models
- neural scaling laws
Sponsors
Notion
More episodes of Machine Learning Street Talk (MLST)
- AI 2040: Plan A report - Daniel Kokotajlo & Thomas Larsen
- Designing How AI Grows — Tom McGrath
- Stealing Reasoning Traces from Proprietary LLM APIs — Ilia Shumailov & Alexander Panfilov
- Every Exponential Ends — Silicon Valley Forgot — Adam Becker
- How Researchers Test AI for Hidden Goals — Apollo Research
Explore listener stats, chart rankings, contacts and more on the Machine Learning Street Talk (MLST) podcast page.