An AI stack: from scaling AI workloads to evaluating LLMs

An AI stack: from scaling AI workloads to evaluating LLMs

From Strachey Lectures by Oxford University

February 26, 2026 · 56 min

About this episode

Professor Ion Stoica discusses scaling AI workloads and evaluating large language models in the context of recent advancements and challenges.

Hilary Term 2026 Strachey Lecture with Professor Ion Stoica, An AI stack: from scaling AI workloads to evaluating LLMs Large language models (LLMs) have taken the world by storm, enabling new applications, intensifying GPU shortages, and raising concerns about the accuracy of their outputs. In this talk, I will present several projects I have worked on to address these challenges. Specifically, I will focus on Ray, a distributed framework for scaling AI workloads, vLLM and SGLang, two high-throughput inference engines for LLMs, and LMArena, a platform for accurate LLM benchmarking. I will conclude with key lessons learned and outline directions for future research.

People in this episode

Guest: Professor Ion Stoica

Topics covered

  • AI workloads
  • large language models
  • GPU shortages
  • inference engines
  • LLM benchmarking
  • future research

Keywords

  • AI stack
  • large language models
  • scaling AI
  • GPU shortages
  • LLM benchmarking
  • Ray
  • vLLM
  • SGLang
  • LMArena

Mentioned in this episode

Organizations: Ray, vLLM, SGLang, LMArena

More episodes of Strachey Lectures

Explore listener stats, chart rankings, contacts and more on the Strachey Lectures podcast page.