Building a custom Tableau provider for Airflow at JLR
About this episode
Najeeb Sulaiman discusses how JLR uses Airflow to orchestrate data and the development of a custom Tableau provider.
JLR is the UK's largest automotive manufacturer, behind brands like Range Rover, Jaguar, Defender, and Discovery. In this episode, Najeeb Sulaiman, Senior Data Engineer at JLR, walks through how Airflow orchestrates data across manufacturing, supply chain, and finance — including a custom Tableau provider his team built (after the community version dropped PAT authentication) and a CI/CD pipeline that validates DAGs before they reach production. Key Takeaways: 00:00 Introduction. 00:48 What JLR makes: luxury vehicles under the Range Rover, Jaguar, Defender, and Discovery brands. 01:42 Najeeb's team in the Data and AI Office, supporting manufacturing, supply chain, finance, and commerce analytics. 03:25 Airflow as the central nervous system of the JLR data stack — the orchestrator that connects every source and downstream system. 05:01 How JLR uses Tableau, and the two modes for getting data in: live connection and scheduled extract refresh. 06:24 Why scheduled Tableau refreshes go stale: they aren't aware of when the data pipeline actually finished. 08:09 First attempt at solving it: Python scripts calling the Tableau REST API directly. 08:47 Why the script approach didn't scale…
People in this episode
Guest: Najeeb Sulaiman
Topics covered
- data engineering
- Apache Airflow
- Tableau integration
- automotive industry
- CI/CD pipelines
- data orchestration
Mentioned in this episode
Organizations: JLR, Tableau, Airflow
Places: UK, Range Rover, Jaguar, Defender, Discovery
More episodes of The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
- Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira
- Enhancing DAGs for Data Processing with William Orgertrice III at Cargill
- Getting Into Data Engineering with Shrividya Hegde, Data and AI Engineer
Explore listener stats, chart rankings, contacts and more on the The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI podcast page.