Spark, AI, and the Future of Data Engineering with Daniel Aronovich
From Data Engineering Central Podcast by Data Engineering in Real Life
March 24, 2026 · 47 min
About this episode
The episode features a discussion with Daniel Aronovich about Apache Spark, the future of data engineering, and the impact of AI on the field.
In this episode of Data Engineering Central , I sit down with the founder of DataFlint , Daniel Aronovich , to talk about the realities of working with Apache Spark, distributed data systems, and the future of data engineering . We start with his early journey into tech—how he first discovered large-scale data systems and the lessons he learned from working with real-world Spark workloads. * The conversation then turns toward the future of data engineering , particularly the growing role of AI in software development and data infrastructure . We discuss why generic AI coding assistants often struggle with complex distributed systems, whether AI will eventually be able to automatically optimize data pipelines, and how the role of the data engineer may evolve in the coming years. We covered a lot of career advice for new and upcoming data professionals. We also discuss the origin of DataFlint , a tool designed to help engineers better understand and optimize Spark workloads by analyzing execution plans, logs, and runtime context. If you work with Spark, large-scale data pipelines, or modern data platforms , this conversation will give you a deeper look into how the data engineering…
People in this episode
Host: Data Engineering Central
Guest: Daniel Aronovich
Topics covered
- Apache Spark
- data engineering
- AI in software development
- distributed data systems
- career advice
Keywords
- data engineering
- Apache Spark
- AI
- data pipelines
- DataFlint
Mentioned in this episode
Organizations: DataFlint, Apache Spark
More episodes of Data Engineering Central Podcast
- From Failure to AWS: What Actually Makes a Great Engineer · June 10, 2026 · 52 min
- How Real Data Engineers Think (Beyond Tools and Hype) · June 3, 2026 · 49 min
- Data, AI, and DuckDB · May 27, 2026 · 50 min
- Why I Left Facebook to Work for Myself · May 20, 2026 · 53 min
- Academic → CTO: What Actually Matters in Data (Matthew Housley) · May 13, 2026 · 56 min
- AI Isn’t Replacing Curious Developers · May 6, 2026 · 1h 4m
Explore listener stats, chart rankings, contacts and more on the Data Engineering Central Podcast podcast page.