Open Source Airflow Contributions and Performance Improvements at G-Research with Christos Bisias
From The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI by Astronomer
March 19, 2026 · 18 min · Season 1 · Episode 73
About this episode
The episode discusses open source contributions and performance improvements in Apache Airflow with guest Christos Bisias from G-Research.
Modern Airflow isn’t just orchestration. It's a contribution. In this episode, we explore how open source investment drives real performance gains and deeper observability. We’re joined by Christos Bisias , Open Source Software Engineer , Apache Airflow at G-Research , to discuss how his team uses Airflow for large-scale data transformations, contributes upstream and improves scheduler throughput and OpenTelemetry support. From trace-level observability to CI-enforced metrics governance and a major scheduler optimization, this conversation spans strategy, engineering and community impact. Key Takeaways: 00:00 Introduction. 01:20 How G-Research applies machine learning and big data to predict financial market movements. 02:15 Contributing to open source is a business decision. 03:10 Maintaining a fork is costly. 04:30 OpenTelemetry collects metrics, logs and traces to provide deep system visibility. 06:10 Custom spans help identify bottlenecks inside tasks and enable performance optimization. 08:05 OpenTelemetry integration works properly in Airflow 3.0 and above. 10:00 A YAML-based metrics registry with CI enforcement ensures consistency between docs and exported metrics. 12:10…
People in this episode
Guest: Christos Bisias
Topics covered
- open source contributions
- performance improvements
- data engineering
- Apache Airflow
- scheduler optimization
- observability
Keywords
- open source
- Airflow
- performance gains
- scheduler throughput
- OpenTelemetry
- data transformations
- machine learning
Mentioned in this episode
Organizations: G-Research, OpenTelemetry
Products: Apache Airflow
More episodes of The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
- Managing a Customer Analytics Platform with Airflow at Skimlinks · June 11, 2026 · 23 min
- Building a custom Tableau provider for Airflow at JLR · June 4, 2026 · 21 min
- Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira · May 28, 2026 · 33 min
- Enhancing DAGs for Data Processing with William Orgertrice III at Cargill · May 21, 2026 · 26 min
- Getting Into Data Engineering with Shrividya Hegde, Data and AI Engineer · May 14, 2026 · 28 min
- Orchestrating DBT With Cosmos and Airflow with Filip Kunčar at ShipMonk Product Development · May 7, 2026 · 25 min
Explore listener stats, chart rankings, contacts and more on the The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI podcast page.