Inference in Action: Scaling Al Smarter with Inferless
From Kubernetes Bytes by Ryan Wallner & Bhavin Shah
October 24, 2024 · 55 min · Season 4 · Episode 20
About this episode
The episode features Nilesh Agarwal discussing Inferless and its impact on serverless GPU inference and model deployment.
In this episode, we sit down with Nilesh Agarwal, co-founder of Inferless, a platform designed to streamline serverless GPU inference. We'll cover the evolving landscape of model deployment, explore open-source tools like KServe and Knative, and discuss how Inferless solves common bottlenecks, such as cold starts and scaling issues. We also take a closer look at real-world examples like CleanLab, who saved 90% on GPU costs using Inferless. Whether you're a developer, DevOps engineer, or tech enthusiast curious about the latest in AI infrastructure, this podcast offers insights into Kubernetes-based model deployment, efficient updates, and the future of serverless ML. Tune in to hear Nilesh's journey from Amazon to founding Inferless and how his platform is transforming the way companies deploy machine learning models. Subscribe now for more episodes! Show Links: * OpenShift 4.17 is GA https://www.youtube.com/live/DvKHwz-c11c?si=6Zap6hk_GsQfdX2m * Policy SBOM from Styra: https://www.styra.com/blog/introducing-policy-sbom/ * NVIDIA GEForce NOW runs on KubeVirt https://thenewstack.io/now-nvidia-scaled-its-cloud-services-with-kubevirt/ * CBT feedback…
People in this episode
Hosts: Ryan Wallner, Bhavin Shah
Guest: Nilesh Agarwal
Topics covered
- AI infrastructure
- serverless GPU inference
- model deployment
- Kubernetes
- scaling issues
- open-source tools
- machine learning
Keywords
- Inferless
- GPU inference
- Kubernetes
- model deployment
- serverless ML
- scaling
- cold starts
- open-source tools
- CleanLab
- AI
Mentioned in this episode
Organizations: Inferless, Amazon, CleanLab, KServe, Knative, OpenShift, NVIDIA, KubeVirt, CNCF, Palumi
More episodes of Kubernetes Bytes
- Building Grafana Labs and the Future of Observability with Anthony Woods · May 29, 2026 · 58 min
- Kubernetes for VMware Admins: Understanding KubeVirt · March 6, 2026 · 58 min
- Secure your Kubernetes applications with Chainguard · February 9, 2026 · 41 min
- Diving Into Kubernetes: The Developer's First Steps with New Relic · March 25, 2025 · 52 min
- Database as a service with Percona Everest · March 3, 2025 · 1h 3m
- KubeCon NA 2024 News Recap · December 18, 2024 · 58 min
Explore listener stats, chart rankings, contacts and more on the Kubernetes Bytes podcast page.