About this episode

Robert Nishihara discusses maximizing hardware utilization for AI and data-intensive workloads using Ray and Kubernetes.

Summary In this episode Robert Nishihara, co-founder of Anyscale and co-creator of Ray, talks about maximizing hardware utilization for AI and data-intensive workloads. He explores Ray’s evolution alongside Kubernetes and PyTorch, and why consolidation at these layers has enabled a new generation of complex, heterogeneous workloads. Robert explains how data preparation has shifted to GPU- and inference-heavy, multimodal pipelines; where Ray fits compared to Spark and workflow orchestrators; and why Ray excels at composing heterogeneous pools of compute, handling failures, and scaling complex systems like multi-node LLM inference and reinforcement learning. He digs into practical strategies for boosting GPU utilization across training and inference, elasticity and prioritization of workloads, topology-aware scheduling, and the importance of fast failure recovery as hardware scales from nodes to racks. If you’re wrestling with expensive GPUs, multimodal data curation, or cross-node LLM inference, this conversation offers concrete mental models and architectural guidance. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Your host…

People in this episode

Host: Tobias Macey

Guest: Robert Nishihara

Topics covered

• GPU utilization
• heterogeneous pipelines
• AI workloads
• data preparation
• Kubernetes
• Ray
• workflow orchestration

Keywords

• GPU
• AI
• data-intensive workloads
• Ray
• Kubernetes
• multimodal pipelines
• workflow orchestration
• failure recovery
• elasticity
• scaling

Mentioned in this episode

Organizations: Anyscale, Ray, Kubernetes, PyTorch, Spark, LLM

Explore listener stats, chart rankings, contacts and more on the Data Engineering Podcast podcast page.