Managing Data Quality and Governance With Airflow at Credit Karma with Ashir Alam
From The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI by Astronomer
March 26, 2026 · 22 min · Season 1 · Episode 74
About this episode
Ashir Alam discusses managing data quality and governance at Credit Karma using Airflow.
Data quality is not optional when you manage credit data at scale. In this episode, Ashir Alam , Senior Data Engineer at Credit Karma , joins us to share how his team acts as the gatekeeper for credit data ingestion, how they standardize data quality with Airflow and DAG Factory and how they scale safely across thousands of DAGs. We explore how governance, PII protection and orchestration come together inside a modern data platform. Key Takeaways: 00:00 Introduction. 01:00 Overview of Credit Karma’s products and financial data ecosystem. 02:00 The team acts as gatekeepers for ingesting data from TransUnion and Equifax. 03:00 Why PII handling and controlled downstream access led to adopting Airflow. 04:00 BigQuery as the warehouse and Airflow as the primary orchestrator. 05:00 Why data quality and governance are critical in financial systems. 07:00 Why Airflow was selected: ease of use and unified ETL plus data quality. 09:00 Introduction to DAG Factory and YAML-based DAG generation. 10:00 GitHub executor creates PR-driven DAG workflows with CI checks. 12:00 BigQuery operators, structured checks and custom Slack and PagerDuty alerts. 13:00 Failed checks stop ETL pipelines and…
People in this episode
Guest: Ashir Alam
Topics covered
- data quality
- data governance
- credit data management
- Airflow orchestration
- DAG scaling
- financial systems
Keywords
- data quality
- data governance
- Airflow
- DAG Factory
- credit data
- financial systems
- BigQuery
- PII protection
Mentioned in this episode
Organizations: Credit Karma, TransUnion, Equifax, BigQuery
Products: Airflow, DAG Factory
More episodes of The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
- Managing a Customer Analytics Platform with Airflow at Skimlinks · June 11, 2026 · 23 min
- Building a custom Tableau provider for Airflow at JLR · June 4, 2026 · 21 min
- Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira · May 28, 2026 · 33 min
- Enhancing DAGs for Data Processing with William Orgertrice III at Cargill · May 21, 2026 · 26 min
- Getting Into Data Engineering with Shrividya Hegde, Data and AI Engineer · May 14, 2026 · 28 min
- Orchestrating DBT With Cosmos and Airflow with Filip Kunčar at ShipMonk Product Development · May 7, 2026 · 25 min
Explore listener stats, chart rankings, contacts and more on the The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI podcast page.