Under which category would you file this issue?
Airflow Core
Apache Airflow version
3.2.0
What happened and how to reproduce it?
We set max_active_runs_per_dag = 1 in airflow.cfg under [core] so that DAGs default to sequential execution. DAGs that need concurrency set max_active_runs explicitly in their definition.
The problem: if a DAG sets max_active_runs=16, the effective value ends up being 1 - the config value - in the database. It looks like 16 is being treated as "not explicitly set" because it matches Airflow's built-in default for max_active_runs.
Setting max_active_runs=17 or any other value that isn't 16 works fine, Airflow respects it.
- Set
max_active_runs_per_dag = 1 in airflow.cfg.
- Create a DAG with
max_active_runs=16:
@dag(
dag_id="test-concurrent",
schedule=None,
max_active_runs=16,
)
def my_dag():
...
- Let the scheduler pick it up. Check the effective
max_active_runs in the metadata database or the DAG details page in the UI. It will show 1 instead of 16.
- Change it to
max_active_runs=17. Now it correctly shows 17.
What you think should happen instead?
If a DAG explicitly sets max_active_runs=16, that should take precedence over max_active_runs_per_dag in airflow.cfg, even though 16 happens to match Airflow's built-in default.
Operating System
Debian GNU/Linux 12 (bookworm) official apache/airflow:3.2.0 container image
Deployment
Other Docker-based deployment
Apache Airflow Provider(s)
No response
Versions of Apache Airflow Providers
No response
Official Helm Chart version
Not Applicable
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
Our workaround is setting max_active_runs to the desired value + 1 for any DAG that needs concurrency. Not ideal but it works.
Are you willing to submit PR?
Code of Conduct
Under which category would you file this issue?
Airflow Core
Apache Airflow version
3.2.0
What happened and how to reproduce it?
We set
max_active_runs_per_dag = 1inairflow.cfgunder[core]so that DAGs default to sequential execution. DAGs that need concurrency setmax_active_runsexplicitly in their definition.The problem: if a DAG sets
max_active_runs=16, the effective value ends up being1- the config value - in the database. It looks like16is being treated as "not explicitly set" because it matches Airflow's built-in default formax_active_runs.Setting
max_active_runs=17or any other value that isn't 16 works fine, Airflow respects it.max_active_runs_per_dag = 1inairflow.cfg.max_active_runs=16:max_active_runsin the metadata database or the DAG details page in the UI. It will show1instead of16.max_active_runs=17. Now it correctly shows17.What you think should happen instead?
If a DAG explicitly sets
max_active_runs=16, that should take precedence overmax_active_runs_per_daginairflow.cfg, even though16happens to match Airflow's built-in default.Operating System
Debian GNU/Linux 12 (bookworm) official apache/airflow:3.2.0 container image
Deployment
Other Docker-based deployment
Apache Airflow Provider(s)
No response
Versions of Apache Airflow Providers
No response
Official Helm Chart version
Not Applicable
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
Our workaround is setting
max_active_runsto the desired value + 1 for any DAG that needs concurrency. Not ideal but it works.Are you willing to submit PR?
Code of Conduct