AI Has a Security Measurement Problem | Gary McGraw

About this episode

Gary McGraw discusses AI security benchmarks and the application of software security principles to AI systems.

Few people (if any) have spent more time thinking about and working on the hard problems in security and software than Gary McGraw, and he also happens to have a PhD in cognitive science and computer science and has been studying neural nets and AI systems for 30+ years. Gary joins Dennis to talk about his team's new research into AI security benchmarks, measurement, and bringing a software security approach to LLMs and AI systems. Links BIML report : https://berryvilleiml.com/results/no-security-meter-ai.pdf

People in this episode

Host: Dennis

Guest: Gary McGraw

Topics covered

• AI security
• software security
• neural networks
• AI benchmarks
• LLMs

Keywords

• AI security
• software security
• neural networks
• AI benchmarks
• LLMs
• measurement
• cognitive science

Mentioned in this episode

Organizations: BIML

More episodes of Decipher Security Podcast

• The Shrinking Exploit Window, Patch Schedule Changes, and the Vulnpocalypse · June 12, 2026 · 33 min
• How The Conversation Predicted Our Surveillance Society 50 Years Ago · June 8, 2026 · 59 min
• Shai Hulud Returns, How Attackers are Using AI, and More Weird MSRC Behavior · June 5, 2026 · 37 min
• Microsoft Has Forgotten Its Vulnerability Disclosure History · May 29, 2026 · 46 min
• Lessons in Resilience, Perseverance, and Leadership With Matt Eversmann · May 25, 2026 · 1h 18m
• Chain Chain Chain of Compromises · May 22, 2026 · 22 min