Ep 89: AI Research Legend’s Honest Assessment of Where We Are
From Unsupervised Learning with Jacob Effron by by Redpoint Ventures
June 3, 2026 · 1h 14m · Episode 89
About this episode
Lukasz Kaiser discusses the limits of current AI architectures and the future of transformers in AI research.
This episode with Lukasz Kaiser, co-author of the seminal "Attention Is All You Need" transformer paper and former researcher at both Google Brain and OpenAI, is a wide-ranging conversation about the fundamental limits of current AI architectures and whether transformers will continue to dominate or eventually give way to something new. Lukasz brings a rare dual perspective: deep belief in how far the current paradigm has taken us (he's an enthusiastic daily Codex user who's seen 10x productivity gains in his own research), while maintaining genuine intellectual humility about whether transformers can truly generalize the way humans do. The episode weaves together questions about data efficiency, the non-verifiable RL frontier, the coding agent revolution, the open vs. closed source gap, and what the next architectural leap might look like: all filtered through the lens of someone who helped build the foundation the entire field is standing on.
People in this episode
Host: Jacob Effron
Guest: Lukasz Kaiser
Topics covered
- AI architectures
- transformers
- data efficiency
- reinforcement learning
- coding agents
- architectural leaps
Keywords
- AI
- transformers
- Lukasz Kaiser
- Google Brain
- OpenAI
- Attention Is All You Need
- data efficiency
- reinforcement learning
Mentioned in this episode
Organizations: Google Brain, OpenAI
Books & works: Attention Is All You Need
More episodes of Unsupervised Learning with Jacob Effron
- AI Vibe Check: Lab Wars, Why APIs Might Vanish & Future Predictions · June 12, 2026 · 1h 7m
- Ep 88: Unpacking DeepMind's Quest for SuperIntelligence with Demis Hassabis' Biographer · June 1, 2026 · 56 min
- Ep 87: Gemini Co-Lead on World Models, RL's Next Domains & Continual Learning · May 22, 2026 · 60 min
- Ep 86: Yann LeCun on Leaving Meta, Breaking The LLM Paradigm, & Why Hinton is Wrong · May 15, 2026 · 1h 22m
- Ep 85: Has AI Infra Stabilized, FM Vibe Shift, & What's Next for Coding Agents · April 23, 2026 · 55 min
- Ep 84: OpenAI’s Chief Scientist on Continual Learning Hype, RL Beyond Code, & Future Alignment Directions · April 9, 2026 · 59 min
Explore listener stats, chart rankings, contacts and more on the Unsupervised Learning with Jacob Effron podcast page.