A deep dive on AI model distillation attacks
About this episode
James Wilson explores AI model distillation attacks and their implications for competitive advantage in LLM-based products.
In this solo episode of Risky Business Features James Wilson explores how distillation techniques are both a legitimate way to train smaller models, as well as a way to steal model capabilities. It’s not just a problem for frontier labs! Any LLM-based product could have its competitive advantage stolen through these attacks. James covers: High-level concept of distillation Why it matters including close/open-weight/open-source explanation Types of distillation and the prompts used The distillation pipeline end to end Distillation at scale and mitigation techniques Hardware resource constraints for distillation
People in this episode
Host: James Wilson
Topics covered
- AI model distillation
- model capabilities theft
- LLM-based products
- distillation techniques
- mitigation techniques
Keywords
- AI
- model distillation
- attacks
- LLM
- competitive advantage
- mitigation
- distillation pipeline
More episodes of Risky Business Features
- Mythos smythos! How to find 0day with lesser models
- Feature Interview: Nicholas Carlini, Anthropic
- A builder's perspective on Mythos and frontier models
- Mythos and 0day: Fixing exploits is not safety
- Mythos and 0day: A hacker’s perspective
- What happens after North Korea infiltrates?
Explore listener stats, chart rankings, contacts and more on the Risky Business Features podcast page.