41 - Lee Sharkey on Attribution-based Parameter Decomposition

About this episode

Daniel Filan chats with Lee Sharkey about Attribution-based Parameter Decomposition and its implications for interpretability in neural networks.

What's the next step forward in interpretability? In this episode, I chat with Lee Sharkey about his proposal for detecting computational mechanisms within neural networks: Attribution-based Parameter Decomposition, or APD for short. Patreon: https://www.patreon.com/axrpodcast Ko-fi: https://ko-fi.com/axrpodcast Transcript: https://axrp.net/episode/2025/06/03/episode-41-lee-sharkey-attribution-based-parameter-decomposition.html Topics we discuss, and timestamps: 0:00:41 APD basics 0:07:57 Faithfulness 0:11:10 Minimality 0:28:44 Simplicity 0:34:50 Concrete-ish examples of APD 0:52:00 Which parts of APD are canonical 0:58:10 Hyperparameter selection 1:06:40 APD in toy models of superposition 1:14:40 APD and compressed computation 1:25:43 Mechanisms vs representations 1:34:41 Future applications of APD? 1:44:19 How costly is APD? 1:49:14 More on minimality training 1:51:49 Follow-up work 2:05:24 APD on giant chain-of-thought models? 2:11:27 APD and "features" 2:14:11 Following Lee's work Lee links (Leenks): X/Twitter: https://twitter.com/leedsharkey Alignment Forum: https://www.alignmentforum.org/users/lee_sharkey Research we discuss: Interpretability in Parameter Space: Minimizing…

People in this episode

Host: Daniel Filan

Guest: Lee Sharkey

Topics covered

• interpretability
• neural networks
• Attribution-based Parameter Decomposition
• computational mechanisms
• hyperparameter selection
• future applications

Keywords

• Attribution-based Parameter Decomposition
• APD
• interpretability
• neural networks
• computational mechanisms
• hyperparameter selection
• minimality
• faithfulness

Mentioned in this episode

Organizations: Patreon, Ko-fi

Books & works: Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-Based Parameter Decomposition, Toy Models of Superposition, Towards a unified and verified understanding of group-operation networks

More episodes of AXRP - the AI X-risk Research Podcast

• 49 - Caspar Oesterheld on Program Equilibrium · February 18, 2026 · 2h 32m
• 48 - Guive Assadi on AI Property Rights · February 15, 2026 · 2h 6m
• 47 - David Rein on METR Time Horizons · January 2, 2026 · 1h 47m
• 46 - Tom Davidson on AI-enabled Coups · August 7, 2025 · 2h 5m
• 45 - Samuel Albanie on DeepMind's AGI Safety Approach · July 6, 2025 · 1h 16m
• 44 - Peter Salib on AI Rights for Human Safety · June 28, 2025 · 3h 22m