Apache Airflow is a widely used open source tool in organisations with large amounts of data processing. Created by AirBnb in 2015, Airflow is highly extensible, supporting many use cases in data engineering. It is desgined to orchestrate your data pipelines which are defined by directed acyclic graphs(DAGs).

There are several ways to install Python on your system and each come with their advantages and disadvantages. For example, a data scientist will benefit from the pre-installed packages like SciPy and Numpy from a quick Anaconda installation. For developers however, it’s better to use tools that provide ease of switching between different versions of Python as your projects may require specific versions.For example, a data scientist will benefit from the pre-installed packages like SciPy and Numpy from a quick Anaconda installation.