Tracking Drift to Monitor LLM Performance

About this episode

This episode discusses monitoring the performance of Large Language Models in production and the importance of drift monitoring.

In this episode, we discuss how to monitor the performance of Large Language Models (LLMs) in production environments. We explore common enterprise approaches to LLM deployment and evaluate the importance of monitoring for LLM quality or the quality of LLM responses over time. We discuss strategies for "drift monitoring" — tracking changes in both input prompts and output responses — allowing for proactive troubleshooting and improvement via techniques like fine-tuning or augmenting data sources. Read the article by Fiddler AI and explore additional resources on how AI observability can help developers build trust into AI services.

Topics covered

• LLM performance
• drift monitoring
• AI observability
• enterprise AI
• proactive troubleshooting

Keywords

• LLM
• monitoring
• drift
• performance
• AI
• enterprise
• troubleshooting

Mentioned in this episode

Organizations: Fiddler AI

Products: Large Language Models

More episodes of Safe and Sound AI

• The Anatomy of Agentic Observability · October 28, 2025 · 16 min
• Agentic Observability: The AI Architect's Essential Blueprint · July 24, 2025 · 14 min
• How to Identify ML Drift Before You Have a Problem · May 31, 2025 · 9 min
• Industry’s Fastest Guardrails Now Native to NVIDIA NeMo · April 2, 2025 · 10 min
• Introducing Fiddler Guardrails: The Fastest in the Industry · February 27, 2025 · 7 min
• Should you Observe ML Metrics or Inferences? · February 12, 2025 · 13 min