Helm Chart for Apache Airflow

Apache Airflow is a platform to programmatically author, schedule and monitor workflows.

Introduction

This chart will bootstrap an Airflow deployment on a Kubernetes cluster using the Helm package manager. This deployment includes custom defined PVC and PV for logs storage. It syncs in DAGs from a GitHub repository.

Requirements

• Kubernetes 1.23+ cluster
• Helm 3.0+
• PV provisioner support in the underlying infrastructure (optionally)

Features

• Supported executors: LocalExecutor, CeleryExecutor, KubernetesExecutor, LocalKubernetesExecutor, CeleryKubernetesExecutor
• Supported Airflow version: 1.10+, 2.0+
• Supported database backend: PostgresSQL, MySQL
• Autoscaling for CeleryExecutor provided by KEDA
• PostgreSQL and PgBouncer with a battle-tested configuration
• Monitoring:
  • StatsD/Prometheus metrics for Airflow
  • Prometheus metrics for PgBouncer
  • Flower
• Automatic database migration after a new deployment
• Administrator account creation during deployment
• Kerberos secure configuration
• One-command deployment for any type of executor. You don't need to provide other services e.g. Redis/Database to test the Airflow.

Documentation

Full documentation for Helm Chart (latest stable release) lives on the website.