Airflow Tutorial (with docker setup)
Concepts.md - Contains all the concpets of airflow platform.
Architectural-View.md - Airflow Architect component concepts.
Working-&-Config.md - Airflow working & configurable parameters concepts.
SetUp Airflow
- create & activate conda env:
airflow_tut cd airflow-Tut- Use Bash script to easily Setup & Run airflow using docker
Run airflow ./airflow-docker-setup-bash-script/airflow-docker-setup.sh
Down/close the aiflow docker compose, also all the Volumnes (-V)
sudo docker-compose down -v
If you want to clean the defaults exmaples of airflow, jusmake "AIRFLOW__CORE__LOAD_EXAMPLES" as "False"
Next, Initialization of airflow using docker compose..
sudo docker-compose up airflow-init
Now Up/launch the docker compose. and ones this is done you will see that your DAG should be visible on airflow UI web interface.
sudo socker-compose up -d