How private LLM inference actually works
About this episode
James Wilson interviews Tanya Verma about the mechanics and implications of private LLM inference in cloud environments.
In this podcast episode James Wilson chats with Tinfoil co-founder Tanya Verma about how you can run a powerful LLM in the cloud without the inference provider seeing your prompts. Tanya talks James through how private inference works, from trusted execution environments and hardware attestation, to TLS termination and GPU isolation. Customers can verify the exact code and model processing their data, while Tinfoil and its infrastructure providers remain locked out. That’s clever engineering… but who really needs it? Is private inference only useful if you’re doing something bad, or will it become a privacy baseline like TLS? James and Tanya discuss the costs and trade-offs, and how open weights make private inference more transparent and trustworthy.
People in this episode
Host: James Wilson
Guest: Tanya Verma
Topics covered
- private LLM inference
- cloud computing
- trusted execution environments
- hardware attestation
- privacy
- GPU isolation
Keywords
- private inference
- LLM
- cloud
- privacy baseline
- TLS
- GPU isolation
- trusted execution environments
- hardware attestation
Mentioned in this episode
Organizations: Tinfoil
More episodes of Risky Business Features
- Who gets to hack the hackers?
- How Brian Krebs doxxed TeamPCP
- James Kettle on inventing new attack techniques with LLMs
- Benchmarks, borders and the true cost of AI regulation
Explore listener stats, chart rankings, contacts and more on the Risky Business Features podcast page.