Teardown Waiting for All in-scope Tasks to Complete

[huashi-st]

Fix teardown tasks firing before all in-scope tasks complete.

Previously, teardown only checked direct upstream states, so it could run while cleared or parallel in-scope tasks were still executing. adds _evaluate_teardown_scope() to verify all tasks between setup and teardown have reached a terminal state before proceeding.

closes: #29332

---

Test Dag

import time
from datetime import datetime
from airflow.sdk import DAG, task
@task
def setup_resource():
    print("Setting up resource")
@task
def task_fail():
    raise ValueError("I'm designed to fail quickly")
@task
def task_slow():
    print("Starting slow task...")
    time.sleep(120)
    print("Slow task done")
@task
def dummy():
    print("Dummy task")
@task
def teardown_resource():
    print("Tearing down resource")
with DAG(
    dag_id="test_teardown_wait",
    start_date=datetime(2024, 1, 1),
    schedule=None,
    catchup=False,
) as dag:
    s = setup_resource()
    t_fail = task_fail()
    t_slow = task_slow()
    d = dummy()
    td = teardown_resource().as_teardown(setups=s)
    s >> [t_fail, t_slow] >> d >> td

Before (at the time task_fail failed) -

After (at the time task_fail failed) -

Was generative AI tooling used to co-author this PR?

• Yes — Claude (Cursor)

Generated-by: Claude (Cursor) following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[morooshka]

@huashi-st Ask please @jscheffl to review. Nice patch 👍

[Nataneljpwd]

Looks great!
Some minor improvements can be done, otherwise looks great

[Nataneljpwd]

Looks great!

[jscheffl]

Looks good! Am a bit envy because I attempted a fix also (some time ago) and failed in making pytests work. But seems you mastered it! Thanks!

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.