Getting Into Data Engineering with Shrividya Hegde, Data and AI Engineer

About this episode

This episode explores building a career in data engineering and the impact of AI on that path with guest Shrividya Hegde.

In this episode, we take a step back from implementation-specific topics to explore what it actually takes to build a career in data engineering — and how AI is reshaping that path. Shrividya Hegde , a data and AI engineer and an Airflow champion in Astronomer’s Champions program, joins us to discuss getting into data engineering, contributing to open source and why good data engineering should make AI output trustworthy rather than confidently wrong. Key Takeaways: 00:00 Introduction. 04:08 Build fundamentals before chasing trending tools — understanding what a tool does, why it exists and what problem it solves has to come first.  07:19 Data engineering fundamentals mean SQL query performance under joins and aggregations, how data moves between pipelines, DAG failure recovery and idempotency — not just writing queries.  08:10 The most common mistake newer data engineers make is skipping fundamentals to chase trends — it is a sequencing problem, not a talent problem.  13:15 AI creates more opportunity for data engineers because AI output quality is directly determined by the quality of the data pipeline feeding it — confidently wrong output is harder to catch than…

People in this episode

Host: Astronomer

Guest: Shrividya Hegde

Topics covered

• data engineering
• AI
• career development
• open source
• data quality

Keywords

• data engineering
• AI
• Airflow
• open source
• data pipeline

Mentioned in this episode

Organizations: Astronomer

Products: Airflow

More episodes of The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• Managing a Customer Analytics Platform with Airflow at Skimlinks · June 11, 2026 · 23 min
• Building a custom Tableau provider for Airflow at JLR · June 4, 2026 · 21 min
• Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira · May 28, 2026 · 33 min
• Enhancing DAGs for Data Processing with William Orgertrice III at Cargill · May 21, 2026 · 26 min
• Orchestrating DBT With Cosmos and Airflow with Filip Kunčar at ShipMonk Product Development · May 7, 2026 · 25 min
• Building Airflow CTL with Buğra Öztürk at Mollie · April 30, 2026 · 20 min