Conversation
Subham-KRLX
requested review from
amoghrajesh,
ashb,
bugraoz93,
hussein-awala,
jedcunningham,
jscheffl and
kaxil
as code owners
Subham-KRLX
force-pushed
the
fix-dag-test-trigger-404
branch
from
282f82e to
1181e43
Compare
There was a problem hiding this comment.
Pull request overview
Fixes DAG.test() failing with TriggerDagRunOperator due to the in-process Execution API not finding the triggered (target) DAG in the metadata DB, by adjusting how DAG bundles are synchronized during dag.test().
Changes:
- Remove the parsing-context restriction during bundle synchronization in
DAG.test()so more DAGs are serialized/synced to the DB. - Add a regression test ensuring
dag.test()succeeds when a DAG triggers another DAG viaTriggerDagRunOperator. - Add a dedicated unit-test DAG file containing a parent DAG that triggers a target DAG.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| task-sdk/src/airflow/sdk/definitions/dag.py | Adjusts DAG.test() bundle parsing/sync behavior to include triggered DAGs. |
| airflow-core/tests/unit/models/test_dag.py | Adds regression test for dag.test() + TriggerDagRunOperator. |
| airflow-core/tests/unit/dags/test_dag_test_trigger_dagrun_operator.py | Provides parent/target DAG definitions used by the new regression test. |
| sync_bag_to_db(dagbag, bundle.name, bundle.version) | ||
| version = DagVersion.get_version(self.dag_id) | ||
| if version: | ||
| break |
There was a problem hiding this comment.
The bundle sync loop still breaks as soon as the current DAG’s DagVersion is found. That means bundles later in get_all_dag_bundles() will not be parsed/synced, so dag.test() can still hit the same 404 when TriggerDagRunOperator targets a DAG that lives in a different (later) bundle. If the goal is to make triggered DAGs visible to the in-process Execution API, this should continue syncing all configured bundles (or otherwise ensure the target DAG(s) are synced) rather than breaking early.
There was a problem hiding this comment.
This update ensures dag.test() correctly syncs all available bundles instead of stopping early once it finds the parent DAG this fix allows TriggerDagRunOperator to work reliably across different bundles by making sure all target DAGs are materialized in the database before execution.
This fix ensures all configured bundles are synchronized during dag.test() if the parent DAG is missing from the database. This allows TriggerDagRunOperator to find target DAGs even if they reside in different bundles. Also adds a specific multi-bundle regression test to verify this scenario. closes: apache#64884
Subham-KRLX
force-pushed
the
fix-dag-test-trigger-404
branch
from
1181e43 to
348379b
Compare
This PR fixes a bug where TriggerDagRunOperator failed with a 404 Not Found error when executed via dag.test().
The issue was caused by an overly restrictive parsing context in the test() method that limited DAG synchronization to only the specific DAG being tested. This meant that any target DAGs (triggered by the test) were never synchronized to the metadata database, making them invisible to the in-process Execution API.
Changes:
Removed the _airflow_parsing_context_manager restriction from the bundle synchronization loop in dag.test().
This ensures all DAGs in the test environment's bundles are properly synchronized to the DB before execution begins.
closes: #64884
Was generative AI tooling used to co-author this PR?
Yes — Claude (For pr description)