#250 – Toby Ord on where AGI timelines go wrong
About this episode
Toby Ord discusses common misconceptions about AGI timelines and the potential dangers of AI self-improvement.
Both Silicon Valley and the public can’t get enough of ‘AGI timelines.’ But Toby Ord, senior researcher at Oxford’s AI Governance Initiative and author of The Precipice , believes we consistently make big mistakes when thinking about them. He lays out the 14 ways he most often sees people go wrong: Assuming AI research is just hill-climbing Imagining AI research is just programming Forecasting “could” instead of “will” Believing the current benchmark is the last one Extrapolating trends with no clear finish line Assuming inputs keep scaling at the same rate Conflating intelligence with capability Consuming point estimates and discarding the error bars Dismissing dissenting experts Forecasting very different things while using the same words Assuming capabilities arrive together Treating “we don’t know” as permission to carry on as usual Choosing a plan that minimises regret rather than maximises impact Trusting surface model impressiveness In this extended conversation with Rob Wiblin, Toby also explains why he thinks: AI self-improvement is uniquely dangerous in four ways, but also might not even work A ban on superintelligence is possible A US-China treaty on superintelligence…
More episodes of 80,000 Hours Podcast
- Why the intelligence explosion can't happen inside a data centre | Tom Reed
- Inside the first AI-coordinated cyberattack on a real company
- #253 – AI 2027's author returns with a plan to change the ending | Daniel Kokotajlo
- #252 – Owain Evans on accidentally training AI models to be evil
- #251 – The UK's former head AI safety scientist on how to solve alignment before superintelligence arrives | Geoffrey Irving
- What the hell happened with AGI timelines in 2026? – Rob Wiblin
Explore listener stats, chart rankings, contacts and more on the 80,000 Hours Podcast podcast page.