About this episode

Mat Ryer of Grafana Labs discusses the challenges and considerations of AI observability and operating AI in production environments.

This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Mat Ryer of Grafana Labs about AI observability, production agents, evals, telemetry cost, guardrails, and what changes once AI moves beyond demos and into systems teams actually depend on. Mat is Senior Director of AI at Grafana Labs, where he focuses on how AI fits into observability and production systems. We talk about Grafana Assistant, why AI observability is not just logs, latency, and HTTP 200s, and how teams can measure whether agents are actually helping. Mat gets into evals, LLM-as-judge patterns, traces as a way to think about conversations, user feedback, tool choice, model changes, and the cost of collecting new telemetry. We also dig into UX and trust. If an AI assistant gives you a wall of text, you still have to decide whether to believe it. If it can show the graph, deep link into Grafana, apply filters, and expose the source data, that becomes a much more useful operating experience. The big takeaway: start small, enhance workflows you already have, build feedback loops, and treat production AI like something you…

People in this episode

Guest: Mat Ryer

Topics covered

• AI observability
• production systems
• telemetry
• agents
• user feedback
• UX
• trust

Keywords

• AI
• observability
• production
• telemetry cost
• Grafana
• agents
• user feedback
• UX
• trust

Mentioned in this episode

Organizations: Grafana Labs

Products: Grafana Assistant

More episodes of Ship It Weekly - DevOps, SRE, Platform and Cloud Engineering News

• GitHub API Enumeration, Grok Build CLI Data Exposure, AWS Security Hub Network Scanning, AI-Powered Patch Pressure, and Why Visibility Is Not Ownership · July 17, 2026 · 19 min
• EKS Rollbacks, GitHub Actions Supply Chain Attacks, AI Agentjacking, CloudWatch Log Alarms, and Why Safety Nets Don’t Replace Ownership · July 10, 2026 · 19 min
• Ship It Conversations: Evan Phoenix of Miren on Deployment Pain, Terraform, Waypoint, and Better Defaults for Small Teams · July 6, 2026 · 42 min
• Amazon Q CVEs, Hijacked npm and Go Packages, AWS WAF HTTP/2 Issues, Lambda MicroVMs, and Why Execution Is the Boundary Now · July 2, 2026 · 18 min
• Ship It Conversations: Kat Traxler of Vectra AI on AI Security, the Zero-Day Clock, IAM, and Cloud Risk · June 28, 2026 · 43 min
• containerd CRI Vulnerabilities, Datadog PostgreSQL HA on Kubernetes, AWS DevOps Agent with Datadog MCP Server, EKS Control Plane Egress, and Why Users Feel the Wait · June 25, 2026 · 19 min

Explore listener stats, chart rankings, contacts and more on the Ship It Weekly - DevOps, SRE, Platform and Cloud Engineering News podcast page.