The End of Data Leakage
About this episode
The episode discusses challenges in AI model leaderboards, focusing on data leakage and benchmarking innovations.
In this episode, we dive deep into the challenges facing time series AI model leaderboards, from hidden information leakage to the complexities of benchmarking foundation models. I sit down with Marcel Meyer to unpack why traditional approaches fall short and how our new TS Arena leaderboard is setting a new standard for fair, future-proof evaluation. We explore the pitfalls that plague current benchmarks, the surprising ways data contamination can skew results, and the innovative pre-registration protocol we've developed to keep evaluations honest. If you've ever wondered what it takes to build a truly trustworthy AI leaderboard—or why it matters for industry and research alike—this conversation is packed with insights you won't want to miss.
Topics covered
- data leakage
- AI model leaderboards
- benchmarking
- time series AI
- evaluation protocols
Keywords
- data leakage
- AI
- benchmarking
- time series
- evaluation
- trustworthy AI
- data contamination
More episodes of Industrial AI Podcast
- The CAD AI Hype!?
- Causal AI on the shopfloor
- Data, Memory, Gyms and Models - NEURA's strategy
- Autonomous CNC programming!?
- Can AI Agents Really Run the Factory?
- Unlocking Industrial Knowledge with AI Agents
Explore listener stats, chart rankings, contacts and more on the Industrial AI Podcast podcast page.