"Cognitive Security as an AI Safety Cause Area" by jsteinhardt

About this episode

The episode discusses the risks to human cognitive security posed by advanced AI systems.

As AI systems become more capable, the cognitive security of humans will be increasingly at risk. By cognitive security, I mean the ability of humans to maintain control over their beliefs and actions. Cognitive security could be compromised in several ways: AI could become very good at persuading people of arbitrary positions; interacting with AI could lead humans to lose touch with reality; and AIs could become very effective at blackmail or at producing extremely convincing false inform...

People in this episode

Guest: jsteinhardt

Topics covered

• AI safety
• cognitive security
• persuasion
• human beliefs
• reality perception

Keywords

• AI
• cognitive security
• persuasion
• beliefs
• reality
• blackmail
• false information

More episodes of LessWrong (Curated & Popular)

• "PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett · June 12, 2026 · 2 min
• "Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models" by Anders Cairns Woodruff, Francis Rhys Ward, Dewi Gould, Rauno Arike, Jason R Brown, Jo Jiao, wlanderson, ariana_azarbal, harrymayne, Patrick Leask · June 11, 2026 · 10 min
• "Even “illegible” Mythos reasoning traces seem pretty legible" by faul_sname · June 11, 2026 · 8 min
• "Sequent: scale and automation for higher confidence in alignment" by Geoffrey Irving, Alex HT, Jesse Hoogland, Daniel Murfet, Jacob Pfau, Marco Cozzi, Stan van Wingerden · June 10, 2026 · 23 min
• "The Machines Lack Honour" by Raymond Douglas · June 10, 2026 · 20 min
• "My favorite depiction of utopia" by Caleb Biddulph · June 4, 2026 · 58 min