Stealing Reasoning Traces from Proprietary LLM APIs — Ilia Shumailov & Alexander Panfilov
About this episode
Tim Scarfe discusses with Ilia Shumailov and Alexander Panfilov the implications of stealing reasoning traces from proprietary LLM APIs.
Tim Scarfe speaks with Ilia Shumailov and Alexander Panfilov about their paper, Stealing Reasoning Traces from Proprietary LLM APIs.The core bug sounds deceptively simple: providers return encrypted reasoning state so conversations can be resumed or forked. But those blobs can be replayed across users and sibling models. A smaller model can ask the provider to decrypt the trace, then repeat the hidden reasoning in plain text. The discussion covers leaked private data, a broadly reusable jailbreak, poisoned agent traces, chain-of-thought monitoring, responsible disclosure, and possible defenses.Ilia Shumailov is an AI and security researcher, formerly at Google DeepMind, who completed his Cambridge PhD under Ross Anderson. Alexander Panfilov is a PhD researcher at the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems, working on AI safety, adversarial machine learning, and LLM red-teaming. They close by separating the demonstrated jailbreaking threat from ordinary benign distillation, and by arguing for controlled experiments over sweeping claims.---TIMESTAMPS:00:00:00 Intro montage00:01:33 Portable encrypted thought and decoded reasoning00:24:55 How…
People in this episode
Host: Tim Scarfe
Guests: Ilia Shumailov, Alexander Panfilov
Topics covered
- AI security
- proprietary LLM APIs
- reasoning traces
- data privacy
- jailbreaking
- AI safety
- adversarial machine learning
Keywords
- encrypted reasoning
- data leakage
- AI safety
- responsible disclosure
- chain-of-thought monitoring
- jailbreak
- adversarial attacks
Mentioned in this episode
Organizations: Google DeepMind, ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems
Books & works: Stealing Reasoning Traces from Proprietary LLM APIs, Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
More episodes of Machine Learning Street Talk (MLST)
- AI 2040: Plan A report - Daniel Kokotajlo & Thomas Larsen
- Designing How AI Grows — Tom McGrath
- Every Exponential Ends — Silicon Valley Forgot — Adam Becker
- AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart
- How Researchers Test AI for Hidden Goals — Apollo Research
Explore listener stats, chart rankings, contacts and more on the Machine Learning Street Talk (MLST) podcast page.