GPU Clouds, Aggregators, and the New Economics of AI Compute

About this episode

The episode discusses the strategic realities of sourcing and operating GPUs across clouds with insights from Hugo Shi, co-founder and CTO of Saturn Cloud.

Summary  In this episode I sit down with Hugo Shi, co-founder and CTO of Saturn Cloud, to map the strategic realities of sourcing and operating GPUs across clouds. Hugo breaks down today’s provider landscape—from hyperscalers to full-service GPU clouds, bare metal/concierge providers, and emerging GPU aggregators—and how to choose among them based on security posture, managed services, and cost. We explore practical layers of capability (compute, orchestration with Kubernetes/Slurm, storage, networking, and managed services), the trade-offs of portability on “Kubernetes-native” stacks, and the persistent challenge of data gravity. We also discuss current supply dynamics, the growing availability of on-demand capacity as newer chips roll out, and how AMD’s ecosystem is maturing as real competition to NVIDIA. Hugo shares patterns for separating training and inference across providers, why traditional ML is far from dead, and how usage varies wildly across domains like biotech. We close with predictions on consolidation, full‑stack experiences from GPU clouds, financial-style GPU marketplaces, and much-needed advances in reliability for long-running GPU jobs.&nbsp…

People in this episode

Host: Tobias Macey

Guest: Hugo Shi

Topics covered

• GPU clouds
• AI compute economics
• cloud providers
• Kubernetes
• data gravity
• machine learning

Keywords

• GPU
• cloud computing
• Kubernetes orchestration
• data infrastructure
• AI workloads
• machine learning
• biotech

Mentioned in this episode

Organizations: Saturn Cloud, AMD, NVIDIA

Products: Bruin

More episodes of AI Engineering Podcast

• Kubernetes, Compliance, and Control: The Operational Backbone of AI Sovereignty · February 25, 2026 · 1h 1m
• From Blind Spots to Observability: Operationalizing LLM Apps with OpenLit · February 15, 2026 · 51 min
• Taming Voice Complexity with Dynamic Ensembles at Modulate · February 8, 2026 · 59 min
• The Future of Dev Experience: Spotify’s Playbook for Organization‑Scale AI · January 20, 2026 · 56 min
• Generative AI Meets Accessibility: Benchmarks, Breakthroughs, and Blind Spots with Joe Devon · January 5, 2026 · 56 min
• Beyond the Chatbot: Practical Frameworks for Agentic Capabilities in SaaS · December 29, 2025 · 54 min