Skip to content


MWAA Workshop

This example shows how to create CI/CD pipeline for Airflow DAGs



  • Make sure you have AWS CLI installed and configured with an aws account you want to use.

install cdk

npm install -g aws-cdk
cdk --version

create virtualenv

python3 -m venv .venv
source .venv/bin/activate

install requirements

pip install -r requirements.txt


export AWS_PROFILE=<the configuration profile for aws-cli>

How to deploy

Step 1. Create Bucket

cdk deploy AirflowCICD/Bucket

after the bucket created, do follows.

  1. copy the bucket name from the cdk output, and replace YOUR_BUCKET_NAME with it in the resources/dags/

Pic 1.

  1. go to S3 console and create dags and requirements folder in the bucket and upload resources/requirements.txt into requirements folder.

Pic 2.

  1. copy requirements.txt version string, then set shell variable like below.

Pic 3.

export REQUIREMENTS_S3_OBJ_VER=the_version_stirng_you_copied

Step 2. Create VPC

cdk deploy AirflowCICD/Network

Step 3. Create MWAA

cdk deploy AirflowCICD/MWAA

after the deployment completed, do follows.

  1. copy the topic arn from the cdk output, and replace YOUR_TOPIC_ARN with it in the resources/dags/

Pic 4.

  1. SNS Topic Subscription - set subscription for Email. the confirmation is needed.

Step 4. Create CI/CD Pipeline

cdk deploy AirflowCICD/CICD
  1. copy the git clone url from the cdk output, and execute the command to clone, there are three different ways to clone git, choose one that you prefer. In this example, HTTPS way will be used.

Pic 5.

  1. push source code into the repo
cd airflow_dags
mkdir dags
cp rsources/dags/ dags/
git status
git add .
git commit -m "initial commit"
git push
  1. check pipeline

Pic 6.

Step 5. Check Airflow UI and Email

Go to MWAA Console and Open Airflow UI

Pic 7.

Execute whatsnew DAG. To execute the DAG, toggle the pause button.

Pic 8.

Check an Email after the execution completed.

Pic 9.


No description, website, or topics provided.



Code of conduct

Security policy





No releases published


No packages published