"Even “illegible” Mythos reasoning traces seem pretty legible" by faul_sname

About this episode

The episode discusses the implications of illegible reasoning in AI models and the concerns surrounding their internal languages.

The Claude Fable 5/Mythos 5 System Card has a section in which they talk about illegible reasoning, and provide an "extreme" example thereof. Models developing their own uninterpretable, unmonitorable internal language has been a major theoretical concern for a while, and when o3 was released last year with its disclaim overshadow disclaim vantage style word salad CoT, it seemed like the problem had become real and immediate. And yet, since o3, other model families have not appeared to hav...

Topics covered

• illegible reasoning
• internal language
• model development
• theoretical concerns
• AI models

Keywords

• illegible reasoning
• AI models
• internal language
• theoretical concerns
• Mythos 5

Mentioned in this episode

Organizations: o3

Books & works: Claude Fable 5/Mythos 5 System Card

More episodes of LessWrong (Curated & Popular)

• "PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett · June 12, 2026 · 2 min
• "Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models" by Anders Cairns Woodruff, Francis Rhys Ward, Dewi Gould, Rauno Arike, Jason R Brown, Jo Jiao, wlanderson, ariana_azarbal, harrymayne, Patrick Leask · June 11, 2026 · 10 min
• "Sequent: scale and automation for higher confidence in alignment" by Geoffrey Irving, Alex HT, Jesse Hoogland, Daniel Murfet, Jacob Pfau, Marco Cozzi, Stan van Wingerden · June 10, 2026 · 23 min
• "The Machines Lack Honour" by Raymond Douglas · June 10, 2026 · 20 min
• "My favorite depiction of utopia" by Caleb Biddulph · June 4, 2026 · 58 min
• "Announcing the ARC White-Box Estimation Challenge" by Jacob_Hilton · June 3, 2026 · 5 min