Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering
About this episode
Emmett Shear discusses the complexities of AI alignment and the future of AGI development.
A16z Podcast Key Takeaways 1. Alignment is an Ongoing Process, Not a Fixed State: Alignment must be reframed as a continuous, adaptive process rather than a destination Like families constantly reknitting the social fabric The industry wrongly assumes “abstract alignment” to a singular good, when it actually requires aligning to specific, evolving values as society makes new moral discoveries about cooperation 2. Tool vs. Being: The Critical Fork in AGI Development Major: AI labs are fundamentally divided on whether they’re building tools or beings AGI, by definition, will be a being, and applying steering-control paradigms (which work for tools) to beings recreates historical mistakes with entities “like us, but different.” Non-optional steering without reciprocity is the definition of slavery; AGI requires treating systems as teammates, not instruments 3. Current LLMs Lack Coherent Self-Model, Creating Dangerous Dynamics : Today’s chatbots are “dissociative agreeable neurotics” – mirrors that reflect users to themselves without coherent goals or a theory of mind This creates narcissistic feedback loops where users fall in love with their reflections. Solution: train AIs in…
People in this episode
Guest: Emmett Shear
Topics covered
- AI alignment
- AGI development
- machine learning
- social dynamics
- theory of mind
Keywords
- alignment
- AGI
- AI development
- self-model
- social mind
Mentioned in this episode
Organizations: A16z
More episodes of Podcast Notes Playlist: Startup
- Sell the Truth
- 10 contrarian leadership truths every leader needs to hear | Matt MacInnis (Rippling)
- 2754: Stress is Making You Fat! Here's how and how to fix it.
- John Mackey, Whole Foods Market
- Top 10 Lessons from 2025
- The Life of Jesus
Explore listener stats, chart rankings, contacts and more on the Podcast Notes Playlist: Startup podcast page.