Building an Enterprise AI Agent for Healthcare
About this episode
William Horton discusses the development and deployment of the Maven Assistant, an AI agent for healthcare.
Every capability in an agent needs its own evidence and release bar. A model-provider slip, an incorrect tool call, and a wrong fertility-benefits answer should not be held to the same pass rate. William Horton, Staff AI Engineer at Maven Clinic, joined us the day after Maven Assistant reached its first external users. The agent helps members inside Maven Clinic’s women’s and family healthcare platform find providers, manage appointments, navigate Maven, and get basic health information. William had spent much of launch day reading chat traces and turning the surprises into product decisions and tests. William shows how a production failure moves through Maven’s system: the trace becomes a regression case, code handles deterministic checks, and LLM judges cover behavior that cannot be reduced to exact outputs. Human labels calibrate those judges, while the consequence of a wrong answer determines whether the capability ships. You can apply the same release workflow to the agent you are building now. “For a lot of our tool-call evaluation, I’ll accept that it runs ten times and passes nine times. Going for that ten out of ten is just not worth the effort.” — William Horton, Staff…
More episodes of Vanishing Gradients
Explore listener stats, chart rankings, contacts and more on the Vanishing Gradients podcast page.