Skip to content

Fix blank Assets dependency graph from missing Dag nodes#69171

Merged
pierrejeambrun merged 1 commit into
apache:mainfrom
davidnzhang:fix/asset-graph-missing-dag-nodes
Jul 2, 2026
Merged

Fix blank Assets dependency graph from missing Dag nodes#69171
pierrejeambrun merged 1 commit into
apache:mainfrom
davidnzhang:fix/asset-graph-missing-dag-nodes

Conversation

@davidnzhang

@davidnzhang davidnzhang commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

What

The scheduling dependencies endpoint is currently able to return an edge with a dag: endpoint that is never materialised as a node. This appears to be causing problems for the ELK renderer when creating the dependency graph in the Assets UI.

This PR adds a loop to backfill any dag: nodes that are referenced by an edge but are missing from the nodes set.

Why

The scheduling view of the Assets dependency graph fails to render whenever it contains a Dag referenced by another Dag (e.g. via TriggerDagRunOperator or ExternalTaskSensor) and has no scheduling dependencies of its own.

The get_scheduling_dependencies function iterates over each Dag's dependencies to build a list of nodes and edges. A Dag is only materialised as a dag: node if it has at least one dependency of its own, but it is possible for the same Dag to be an endpoint of an edge if it is referenced by another Dag. This leaves a dangling edge whose endpoint is missing from the node set.

Reproduction

Parent Dag

from datetime import datetime

from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.providers.standard.operators.trigger_dagrun import TriggerDagRunOperator
from airflow.sdk import DAG, Asset

PARENT_ASSET = Asset(name="parent_dag_asset", uri="s3://some-bucket/parent_dag_asset")

with DAG(dag_id="parent_dag", schedule=None, start_date=datetime(2021, 1, 1), catchup=False):
    emit = EmptyOperator(task_id="emit", outlets=[PARENT_ASSET])
    trigger = TriggerDagRunOperator(
        task_id="trigger_child_dag",
        trigger_dag_id="child_dag",
        wait_for_completion=False,
    )
    emit >> trigger

Child Dag

from datetime import datetime

from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.sdk import DAG

with DAG(dag_id="child_dag", schedule=None, start_date=datetime(2021, 1, 1), catchup=False):
    EmptyOperator(task_id="some_task")

Before

Scheduling view

image

/ui/dependencies?dependency_type=scheduling Response

{
    "edges": [
        {
            "source_id": "dag:parent_dag",
            "target_id": "asset:1"
        },
        {
            "source_id": "dag:parent_dag",
            "target_id": "trigger:parent_dag:child_dag:trigger_child_dag"
        },
        {
            "source_id": "trigger:parent_dag:child_dag:trigger_child_dag",
            "target_id": "dag:child_dag"
        }
    ],
    "nodes": [
        {
            "id": "dag:parent_dag",
            "label": "parent_dag",
            "type": "dag",
            "team": null
        },
        {
            "id": "asset:1",
            "label": "parent_dag_asset",
            "type": "asset",
            "team": null
        },
        {
            "id": "trigger:parent_dag:child_dag:trigger_child_dag",
            "label": "trigger_child_dag",
            "type": "trigger",
            "team": null
        }
    ]
}

After

Scheduling view
image

/ui/dependencies?dependency_type=scheduling Response

{
    "edges": [
        {
            "source_id": "dag:parent_dag",
            "target_id": "asset:1"
        },
        {
            "source_id": "dag:parent_dag",
            "target_id": "trigger:parent_dag:child_dag:trigger_child_dag"
        },
        {
            "source_id": "trigger:parent_dag:child_dag:trigger_child_dag",
            "target_id": "dag:child_dag"
        }
    ],
    "nodes": [
        {
            "id": "dag:parent_dag",
            "label": "parent_dag",
            "type": "dag",
            "team": null
        },
        {
            "id": "asset:1",
            "label": "parent_dag_asset",
            "type": "asset",
            "team": null
        },
        {
            "id": "trigger:parent_dag:child_dag:trigger_child_dag",
            "label": "trigger_child_dag",
            "type": "trigger",
            "team": null
        },
        {
            "id": "dag:child_dag",
            "label": "child_dag",
            "type": "dag",
            "team": null
        }
    ]
}

Note node set now correctly includes id: dag:child_dag, so edge below is no longer dangling:

{
      "source_id": "trigger:parent_dag:child_dag:trigger_child_dag",
      "target_id": "dag:child_dag"
  }

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Jun 30, 2026
@davidnzhang davidnzhang marked this pull request as ready for review June 30, 2026 09:13
@davidnzhang davidnzhang force-pushed the fix/asset-graph-missing-dag-nodes branch from 2cd09ef to 183e446 Compare June 30, 2026 21:50

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this fix is just papering over a gap in the existing algorithm. It might be that is the only reasonable solution so you are free to push back.

@pierrejeambrun pierrejeambrun added this to the Airflow 3.3.1 milestone Jul 2, 2026
@pierrejeambrun pierrejeambrun merged commit 87bf023 into apache:main Jul 2, 2026
77 checks passed
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants