Listen: How to do AI analysis you can actually trust
From Lenny's Reads by Lenny Rachitsky
February 17, 2026 · 18 min
About this episode
Caitlin Sullivan shares techniques for obtaining trustworthy insights from AI analysis of customer data.
Ever run an AI analysis on customer data, only to discover the numbers were fabricated and the insights completely generic? In this episode, Caitlin Sullivan, a user-research veteran who's trained hundreds of product and research professionals, shares her four prompting techniques for getting trustworthy, actionable insights out of any LLM. After 2,000+ hours of testing customer discovery workflows with AI, she's identified the failure modes that break AI analysis and the reliable fixes for each one. In this episode, you'll learn: • How to catch the two types of AI quote hallucinations • Why AI defaults to useless generic themes and insights • Which LLM is best for analysis work (and which one fabricates the most) • How to turn vague signal into actual decision clarity • The final verification pass that stress-tests everything before it hits a deck Referenced: • Caitlin Sullivan: https://www.linkedin.com/in/caitlindsullivan/ • Claude Code for Customer Insights (Maven course): https://maven.com/caitlin/claude-code-insights • Claude: https://www.anthropic.com/claude • ChatGPT: https://chatgpt.com/ • Gemini: https://gemini.google.com/ • NotebookLM: https://notebooklm.google.com/ •…
People in this episode
Host: Lenny Rachitsky
Guest: Caitlin Sullivan
Topics covered
- AI analysis
- customer data
- trustworthy insights
- LLM techniques
- product research
- user research
Keywords
- AI analysis
- customer insights
- LLM
- prompting techniques
- data verification
- product research
Mentioned in this episode
Organizations: Lenny's Reads, Maven
Products: Claude, ChatGPT, Gemini, NotebookLM, Maze, Whoop
More episodes of Lenny's Reads
- Goodbye, Lenny's Reads! · May 22, 2026 · 1 min
- Listen: Not all AI agents are created equal · April 15, 2026 · 20 min
- Listen: A visual guide to getting out of a creative slump · April 8, 2026 · 8 min
- Listen: OpenClaw: A power user’s guide to the most powerful personal AI tool since ChatGPT · April 1, 2026 · 25 min
- Listen: State of the product job market in early 2026 · March 24, 2026 · 7 min
- Listen: How I built LennyRPG · March 18, 2026 · 20 min
Explore listener stats, chart rankings, contacts and more on the Lenny's Reads podcast page.