Snap’s Secret to Processing 10 Petabytes a Day: GPU-Accelerated Spark | NVIDIA AI Podcast Ep. 298

About this episode

Prudhvi Vatala discusses how Snap modernized its data infrastructure using NVIDIA GPU-accelerated Apache Spark.

Snap processes more than 10 petabytes of experimentation data every single morning—and with NVIDIA GPU-accelerated Apache Spark on Google Cloud, Snap cut job costs by 76%, reduced memory usage by 80%, and eliminated 120 terabytes of disk spill from its pipelines. Prudhvi Vatala, head of engineering platforms at Snap, joins the NVIDIA AI Podcast to break down how he and his team completely modernized data infrastructure for a social platform serving nearly a billion monthly active users—using NVIDIA cuDF plugin (formerly referred to as NVIDIA RAPIDS plugin) for Apache Spark on Google Kubernetes Engine, with zero application code changes. 🔬Topics covered: How Snap runs A/B tests at planetary scale using rigorous statistical methods like heterogeneous treatment effect detection and variance reduction Why Snap reuses idle inference GPUs between 1–5 a.m. for batch data processing—and how it built a Kubernetes-based platform to do it How NVIDIA cuDF delivered 3x+ speedups on join-heavy Spark jobs with no code rewrites The full business impact: 76% cost reduction, 62% fewer cores, 80% less memory, 120 TB of spill eliminated How a three-way partnership between Snap, NVIDIA, and Google…

People in this episode

Guest: Prudhvi Vatala

Topics covered

• data processing
• GPU acceleration
• A/B testing
• cloud computing
• data infrastructure
• cost reduction

Keywords

• Snap
• NVIDIA
• GPU
• Apache Spark
• data processing
• cost reduction
• Kubernetes
• cloud computing
• A/B testing

Mentioned in this episode

Organizations: Snap, NVIDIA, Google Cloud

Products: NVIDIA cuDF plugin, Apache Spark, Google Kubernetes Engine

More episodes of NVIDIA AI Podcast

• How Mistral Is Building Frontier AI for the Enterprise | NVIDIA AI Podcast Ep. 301 · June 10, 2026 · 22 min
• Everyone Can Build a Robot: Open Source Embodied AI With Seeed Studio | NVIDIA AI Podcast Ep. 300 · May 27, 2026 · 29 min
• Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value | NVIDIA AI Podcast Ep. 299 · May 21, 2026 · 33 min
• Harrison Chase of LangChain on Deep Agents, LangSmith, and Earning Trust | NVIDIA AI Podcast Ep. 297 · May 6, 2026 · 25 min
• How Dassault Systèmes Is Building AI That Understands Physics - Ep. 296 · April 29, 2026 · 23 min
• One Brain, Any Robot: Skild AI's Skild Brain Explained - Ep. 295 · April 22, 2026 · 30 min