Eric Jang – Building AlphaGo from scratch

About this episode

Eric Jang discusses building AlphaGo from scratch using modern AI tools and explores its implications for future AI development.

Eric Jang walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends)…

People in this episode

Host: Dwarkesh Patel

Guest: Eric Jang

Topics covered

• AlphaGo
• AI tools
• reinforcement learning
• intelligence explosion
• AI research

Keywords

• AlphaGo
• AI
• reinforcement learning
• self-play
• MCTS
• LLMs
• research automation

Sponsors

Cursor

More episodes of Dwarkesh Podcast

• Alex Imas and Phil Trammell – What remains scarce after AGI? · June 4, 2026 · 1h 16m
• Reiner Pope – Chip design from the bottom up · May 22, 2026 · 1h 21m
• David Reich – Why the Bronze Age was an inflection point in human evolution · May 8, 2026 · 2h 13m
• Reiner Pope – The math behind how LLMs are trained and served · April 29, 2026 · 2h 14m
• Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat · April 15, 2026 · 1h 43m
• Michael Nielsen – How science actually progresses · April 7, 2026 · 2h 3m