This is not an officially supported Google product.
- Join our Slack channel.
The Airflow Operator is still under active development and has not been extensively tested in production environment. Backward compatibility of the APIs is not guaranteed for alpha releases.
- Version >= 1.9 of Kubernetes.
- Uses 1.9 of Airflow (1.10.0rc2+ for k8s executor)
- Uses 4.0.x of Redis (for celery operator)
- Uses 5.7 of MySQL
Get started quickly with the Airflow Operator using the Quick Start Guide
Airflow Operator Overview
Airflow Operator is a custom Kubernetes operator that makes it easy to deploy and manage Apache Airflow on Kubernetes. Apache Airflow is a platform to programmatically author, schedule and monitor workflows. Using the Airflow Operator, an Airflow cluster is split into 2 parts represented by the
AirflowCluster custom resources.
The Airflow Operator performs these jobs:
- Creates and manages the necessary Kubernetes resources for an Airflow deployment.
- Updates the corresponding Kubernetes resources when the
- Restores managed Kubernetes resources that are deleted.
- Supports creation of Airflow schedulers with different Executors
- Supports sharing of the
Checkout out the Design
Managed Airflow solution
Google Cloud Composer is a fully managed workflow orchestration service targeting customers that need a workflow manager in the cloud.