Getting Into Data Engineering with Shrividya Hegde, Data and AI Engineer
About this episode
This episode explores building a career in data engineering and the impact of AI on that path with guest Shrividya Hegde.
In this episode, we take a step back from implementation-specific topics to explore what it actually takes to build a career in data engineering — and how AI is reshaping that path. Shrividya Hegde , a data and AI engineer and an Airflow champion in Astronomer’s Champions program, joins us to discuss getting into data engineering, contributing to open source and why good data engineering should make AI output trustworthy rather than confidently wrong. Key Takeaways: 00:00 Introduction. 04:08 Build fundamentals before chasing trending tools — understanding what a tool does, why it exists and what problem it solves has to come first. 07:19 Data engineering fundamentals mean SQL query performance under joins and aggregations, how data moves between pipelines, DAG failure recovery and idempotency — not just writing queries. 08:10 The most common mistake newer data engineers make is skipping fundamentals to chase trends — it is a sequencing problem, not a talent problem. 13:15 AI creates more opportunity for data engineers because AI output quality is directly determined by the quality of the data pipeline feeding it — confidently wrong output is harder to catch than…
People in this episode
Host: Astronomer
Guest: Shrividya Hegde
Topics covered
- data engineering
- AI
- career development
- open source
- data quality
Mentioned in this episode
Organizations: Astronomer
Products: Airflow
More episodes of The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
- Building a custom Tableau provider for Airflow at JLR
- Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira
- Enhancing DAGs for Data Processing with William Orgertrice III at Cargill
Explore listener stats, chart rankings, contacts and more on the The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI podcast page.