Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources. Kenneth Harris, a NASA veteran who worked on ...

Apache Airflow is one of the world’s most popular open source tools for building and managing data pipelines, with around 16 million downloads per month. Those users will see several compelling new ...

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More The open-source Apache Airflow 2.9 release is out today, providing users ...

Apache Airflow is a great data pipeline as code, but having most of its contributors work for Astronomer is another example of a problem with open source. Depending on your politics, trickle-down ...

In today's data-driven world, unreliable pipelines lead to broken dashboards, late reports, and untrustworthy analytics. Data Engineers face increasing pressure to ensure their data pipelines deliver ...

Organizations have been working on becoming more data-driven for many years at this point, with mixed results. We understand that the value of data is undeniable. However, it has now become more ...

To build data-driven organizations, enterprises have to deal with a plethora of tooling, making data orchestration fundamental. As the driving force behind open-source workflow management platform ...

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Getting data from where it is created to where it can be used effectively ...

The rapidly changing world of data engineering has seen a significant shift with the combination of Apache Spark, Snowflake, and Apache Airflow. This trio allows organizations to build highly ...