About this episode

Katja Grace discusses the existential risks of AI and her research on AI safety and governance.

This and all episodes at: https://aiandyou.net/ . When you stop and really consider the existential risks of AI, it becomes very apparent that we should do something. Yet so few people do. Someone who’s attempting to make up for that is Katja Grace, founder of AI Impacts, a forecasting and risk research project focused on AI safety and governance. Her surveys of the AI community’s evaluations of risk have gotten her named to Time Magazine’s 100 most influential people in AI for 2024 and her papers on those surveys have been some of the most discussed scientific papers of their time. She’s an expert on technological trajectories, model scaling, and anthropic reasoning. We talk about what’s driving Katja to do this work, the fundamental risks that she sees in AI, AI evolving unexpected goals, risks of agents, comparing unemployment risks and like extinction risks, artificial general intelligence, the stunning results of the surveys she’s run on when AI will have serious effects, and… Trojan Horse racing. All this plus our usual look at today's AI headlines! Transcript and URLs referenced at HumanCusp Blog.

People in this episode

Host: aiandyou

Guest: Katja Grace

Topics covered

• AI safety
• existential risks
• AI governance
• technological trajectories
• model scaling
• anthropic reasoning
• AI community

Keywords

• AI risks
• AI safety
• Katja Grace
• AI governance
• forecasting
• anthropic reasoning
• artificial general intelligence
• surveys

Mentioned in this episode

Organizations: AI Impacts, Time Magazine

More episodes of Artificial Intelligence and You

• 318 - Guest: Rana Gujral, AI Author, part 1 · July 20, 2026 · 36 min
• 317 - Special: Is AI In a Bubble? · July 13, 2026 · 27 min
• 316 - Guest: Gašper Beguš, Marine Linguist, part 2 · July 6, 2026 · 29 min
• 315 - Guest: Gašper Beguš, Marine Linguist, part 1 · June 29, 2026 · 34 min
• 314 - Guest: Katja Grace, AI Impact Researcher, part 2 · June 22, 2026 · 30 min

Explore listener stats, chart rankings, contacts and more on the Artificial Intelligence and You podcast page.