"PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett

About this episode

The episode discusses the misconception that many in the AI safety community are focused on alignment, revealing that few actually are.

People often assume that a large fraction of the AI safety community works on alignment. As far as we're aware, this is not true. Most people are not working on making sure superintelligent AIs are aligned with human values or follow human instructions. Currently, the people who work on alignment are roughly: The Alignment Research Center who work on a research bet by Paul ChristianoProbably Sequent who just got announced yesterdaySome scattered people who work at universities or indepen...

People in this episode

Guests: Chi Nguyen, peterbarnett

Topics covered

• AI safety
• alignment
• superintelligent AIs
• human values
• research community

Keywords

• AI alignment
• AI safety
• superintelligence
• research
• human values

Mentioned in this episode

Organizations: Alignment Research Center, Sequent

More episodes of LessWrong (Curated & Popular)

• "Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models" by Anders Cairns Woodruff, Francis Rhys Ward, Dewi Gould, Rauno Arike, Jason R Brown, Jo Jiao, wlanderson, ariana_azarbal, harrymayne, Patrick Leask · June 11, 2026 · 10 min
• "Even “illegible” Mythos reasoning traces seem pretty legible" by faul_sname · June 11, 2026 · 8 min
• "Sequent: scale and automation for higher confidence in alignment" by Geoffrey Irving, Alex HT, Jesse Hoogland, Daniel Murfet, Jacob Pfau, Marco Cozzi, Stan van Wingerden · June 10, 2026 · 23 min
• "The Machines Lack Honour" by Raymond Douglas · June 10, 2026 · 20 min
• "My favorite depiction of utopia" by Caleb Biddulph · June 4, 2026 · 58 min
• "Announcing the ARC White-Box Estimation Challenge" by Jacob_Hilton · June 3, 2026 · 5 min