Skip to content

max_active_runs in DAG definition ignored when value matches built-in default #65307

@seruman

Description

@seruman

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.2.0

What happened and how to reproduce it?

We set max_active_runs_per_dag = 1 in airflow.cfg under [core] so that DAGs default to sequential execution. DAGs that need concurrency set max_active_runs explicitly in their definition.

The problem: if a DAG sets max_active_runs=16, the effective value ends up being 1 - the config value - in the database. It looks like 16 is being treated as "not explicitly set" because it matches Airflow's built-in default for max_active_runs.

Setting max_active_runs=17 or any other value that isn't 16 works fine, Airflow respects it.

  1. Set max_active_runs_per_dag = 1 in airflow.cfg.
  2. Create a DAG with max_active_runs=16:
    @dag(
        dag_id="test-concurrent",
        schedule=None,
        max_active_runs=16,
    )
    def my_dag():
        ...
  3. Let the scheduler pick it up. Check the effective max_active_runs in the metadata database or the DAG details page in the UI. It will show 1 instead of 16.
  4. Change it to max_active_runs=17. Now it correctly shows 17.

What you think should happen instead?

If a DAG explicitly sets max_active_runs=16, that should take precedence over max_active_runs_per_dag in airflow.cfg, even though 16 happens to match Airflow's built-in default.

Operating System

Debian GNU/Linux 12 (bookworm) official apache/airflow:3.2.0 container image

Deployment

Other Docker-based deployment

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

No response

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

Our workaround is setting max_active_runs to the desired value + 1 for any DAG that needs concurrency. Not ideal but it works.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions