The Future of the Lakehouse: Delta Lake, Rust, and Data Platforms at Scale
About this episode
The episode features a discussion with Ethan about modern lakehouse architecture and building scalable data platforms in regulated environments.
In this episode of the Data Engineering Central Podcast, I sit down with Ethan , a maintainer of delta-rs and an expert in modern lakehouse architecture working in the pharmaceutical industry. We discuss Ethan’s journey into tech and data engineering, the evolution of open table formats like Delta Lake and Apache Iceberg, and what it actually takes to build scalable enterprise data platforms in highly regulated environments like big pharma. We also dive into: * delta-rs and the future of Delta Lake outside Spark * Lakehouse architecture and open catalogs * Rust in the modern data ecosystem * Data platform governance and scalability * Enterprise analytics and infrastructure * The future of agentic analytics and AI-enabled data systems * Lessons learned building large-scale data platforms If you’re interested in modern data engineering, open source infrastructure, lakehouses, or the future of analytics engineering, this is a great conversation. Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit…
More episodes of Data Engineering Central Podcast
- What Happens When a Software Engineer Builds a Company Alone?
- The Creator of Pandas on AI, Apache Arrow, and the Future of Software Engineering
- From COBOL to Copilot: 30 Years of Data, BI, and AI with David Langer
- Semantic Layers, Agents, and the Future of Analytics
Explore listener stats, chart rankings, contacts and more on the Data Engineering Central Podcast podcast page.