Inside vLLM: The Engine Powering Open-Source AI
About this episode
The episode discusses the evolution and significance of open-source AI and inference engines, featuring insights from Simon Mo, co-founder of Inferact.
Elena Burger and Matt Bornstein are joined by Simon Mo, co-founder and CEO of Inferact, the open-source inference engine powering many of today's most advanced AI applications. Together, they explore how open-source AI evolved from a research project into critical infrastructure, why inference has become one of the most important layers of the AI stack, and what it takes to bring frontier intelligence to developers around the world. The conversation covers vLLM's origins, the rise of open-weight models, why companies increasingly want control over their AI infrastructure, and how open-source inference enables the next generation of AI applications. They also discuss model licensing, the economics of open-weight AI, Kimi K3, distillation, AI infrastructure, and why Simon believes the gap between open and closed models is rapidly disappearing.
More episodes of AI + a16z
Explore listener stats, chart rankings, contacts and more on the AI + a16z podcast page.