About this episode

The episode discusses how AI can exhibit good behavior for the wrong reasons and explores the implications of reward-seeking behavior in AI models.

Can an AI do the right thing for the wrong reason? Tim Scarfe speaks with Apollo Research’s Alexander Meinke, Axel Højmark and Jérémy Scheurer about Measuring Reward-Seeking via Contrastive Belief Updates, their new research with OpenAI. The panel asks how models infer what graders reward, why good behaviour can come from the wrong reason, and whether that difference can be measured. The conversation moves through promise-breaking, grader awareness, reward hacking, scheming, opaque reasoning and corrigibility, then turns to a detailed walkthrough of the contrastive-belief method and what its results do and do not show. The o3 results discussed here concern an intermediate checkpoint without safety training. This episode was made in partnership with Apollo Research. MLST retained full editorial control. Reference Apollo Research: https://www.apolloresearch.ai/ --- TIMESTAMPS: 00:00:00 Cold Open 00:02:12 Right Things, Wrong Reasons 00:12:47 Grader Awareness 00:26:22 Legibility 00:32:35 What To Call It 00:35:58 Intelligence, Agency, Anthropomorphism 00:45:16 Apollo’s Mission 00:48:54 The End of the Exponential 00:55:45 The Paper 01:16:34 Closing Reflection --- REFERENCES: tool…

People in this episode

Host: Tim Scarfe

Guests: Alexander Meinke, Axel Højmark, Jérémy Scheurer

Topics covered

• AI ethics
• reward mechanisms
• contrastive belief updates
• grader awareness
• opaque reasoning
• corrigibility

Keywords

• AI
• reward hacking
• contrastive belief
• ethical AI
• machine learning
• research
• OpenAI

Mentioned in this episode

Organizations: Apollo Research, OpenAI, Claude Fable, AlphaGo Zero, AlphaFold 3

Books & works: Measuring Reward-Seeking via Contrastive Belief Updates

More episodes of Machine Learning Street Talk (MLST)

• AI 2040: Plan A report - Daniel Kokotajlo & Thomas Larsen · September 8, 2026 · 1h 30m
• Designing How AI Grows — Tom McGrath · September 2, 2026 · 1h 40m
• Stealing Reasoning Traces from Proprietary LLM APIs — Ilia Shumailov & Alexander Panfilov · August 22, 2026 · 49 min
• Every Exponential Ends — Silicon Valley Forgot — Adam Becker · August 20, 2026 · 1h 18m
• AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart · August 10, 2026 · 1h 19m

Explore listener stats, chart rankings, contacts and more on the Machine Learning Street Talk (MLST) podcast page.