Add three-level run_on_latest_version configuration hierarchy

[nathadfield]

Description

Implements a three-level configuration hierarchy for run_on_latest_version to provide flexible control over bundle version selection when creating DAG runs.

This PR adds:

1. Global configuration: [core] run_on_latest_version option in airflow.cfg
2. DAG-level configuration: @dag(run_on_latest_version=True/False) parameter
3. UI integration: Clear/Rerun dialogs with version selection when applicable

The precedence hierarchy is: DAG-level > Global config > System default (False)

Fixes #60887

Motivation

When working with bundle versioning in production, users need flexible control over which bundle version is used when creating DAG runs. Different DAGs may have different requirements:

• Some DAGs should always use the latest bundle version (e.g., data processing pipelines that need latest code)
• Others should use the version they were originally scheduled with (e.g., financial reporting that requires reproducibility)

Previously, this could only be controlled per-operation (trigger/clear). This PR enables configuration at the global and DAG levels for consistent behavior.

Implementation Details

Configuration Hierarchy

# System default (if not configured anywhere)
run_on_latest_version = False

# Global configuration (applies to all DAGs unless overridden)
[core]
run_on_latest_version = True

# DAG-level configuration (highest precedence)
@dag(
    dag_id="my_dag",
    run_on_latest_version=True,  # Overrides global config
    ...
)
def my_dag():
    ...

Resolution Logic

The resolution logic in SerializedDAG._resolve_bundle_version() follows this precedence:

1. Check DAG-level run_on_latest_version parameter
2. If not set, check global [core] run_on_latest_version config
3. If not set, default to False

UI Integration

Created a custom React hook (useRunOnLatestVersion) that:

• Fetches global config and DAG-level settings
• Determines the default checked state of the checkbox based on precedence hierarchy
• Properly handles nullable overrides to preserve the hierarchy

The Clear/Rerun dialogs show a "Run on latest version" checkbox when the current run's bundle version differs from the latest available version. The checkbox's default state (checked/unchecked) is determined by the configuration hierarchy.

Screen.Recording.2026-01-30.at.14.23.09.mov

Breaking Changes

None. The system default remains False, preserving existing behavior.

Tests

# Scheduler Exception Handling (tests/unit/jobs/test_scheduler_job.py)
test_scheduler_create_dag_runs_handles_bundle_version_unavailable
test_scheduler_create_dag_runs_asset_triggered_handles_bundle_version_unavailable

# DAG Model & Configuration Precedence (tests/unit/models/test_dagmodel.py)
test_create_dagrun_with_global_run_on_latest_version
test_create_dagrun_with_dag_level_run_on_latest_version
test_create_dagrun_disable_bundle_versioning_bypasses_logic
test_disable_bundle_versioning[True-some-version-None]
test_disable_bundle_versioning[False-some-version-some-version]

# API Endpoints (tests/unit/api_fastapi/)
test_dag_details_includes_run_on_latest_version  # core_api/routes/public/test_dags.py
test_get_config_with_run_on_latest_version_true  # core_api/routes/ui/test_config.py
test_get_config_with_run_on_latest_version_false  # core_api/routes/ui/test_config.py
test_trigger_dag_run_bundle_version_not_yet_parsed  # execution_api/versions/head/test_dag_runs.py

# Serialization (tests/unit/serialization/test_dag_serialization.py)
test_dag_run_on_latest_version_serialization[None]
test_dag_run_on_latest_version_serialization[True]
test_dag_run_on_latest_version_serialization[False]

Documentation

Updated docs/administration-and-deployment/dag-bundles.rst with:

• Configuration options at all levels
• Precedence hierarchy explanation
• Examples of common use cases
• UI behavior description

Related Issues/PRs

• Fixes Add configuration option for default run_on_latest_version behavior on DAG clears/reruns #60887 - Global configuration for run_on_latest_version
• Related to Add bundle version support to TriggerDagRunOperator #60880 - Bundle version support for TriggerDagRunOperator (will be addressed in separate PR)

Generative AI Usage

Was generative AI tooling used to co-author this PR?
✅ Yes

Generated-by: Claude Sonnet 4.5 following the Airflow contribution guidelines