Beyond the Chatbot: Practical Frameworks for Agentic Capabilities in SaaS

About this episode

Preeti Shukla discusses the integration of agentic capabilities in SaaS platforms and the operational challenges involved.

Summary  In this episode product and engineering leader Preeti Shukla explores how and when to add agentic capabilities to SaaS platforms. She digs into the operational realities that AI agents must meet inside multi-tenant software: latency, cost control, data privacy, tenant isolation, RBAC, and auditability. Preeti outlines practical frameworks for selecting models and providers, when to self-host, and how to route capabilities across frontier and cheaper models. She discusses graduated autonomy, starting with internal adoption and low-risk use cases before moving to customer-facing features, and why many successful deployments keep a human-in-the-loop. She also covers evaluation and observability as core engineering disciplines - layered evals, golden datasets, LLM-as-a-judge, path/behavior monitoring, and runtime vs. offline checks - to achieve reliability in nondeterministic systems.  Announcements  Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with…

People in this episode

Host: Tobias Macey

Guest: Preeti Shukla

Topics covered

• agentic capabilities
• SaaS platforms
• AI agents
• operational realities
• evaluation and observability
• reliability in nondeterministic systems

Keywords

• agentic capabilities
• SaaS
• AI agents
• latency
• data privacy
• evaluation
• observability
• human-in-the-loop

Mentioned in this episode

Organizations: Cash App, Prefect, Google Cloud, AWS, Databrick

More episodes of AI Engineering Podcast

• Kubernetes, Compliance, and Control: The Operational Backbone of AI Sovereignty · February 25, 2026 · 1h 1m
• From Blind Spots to Observability: Operationalizing LLM Apps with OpenLit · February 15, 2026 · 51 min
• Taming Voice Complexity with Dynamic Ensembles at Modulate · February 8, 2026 · 59 min
• GPU Clouds, Aggregators, and the New Economics of AI Compute · January 27, 2026 · 46 min
• The Future of Dev Experience: Spotify’s Playbook for Organization‑Scale AI · January 20, 2026 · 56 min
• Generative AI Meets Accessibility: Benchmarks, Breakthroughs, and Blind Spots with Joe Devon · January 5, 2026 · 56 min