DeadlineAlert records not created for updated DAGs — SLA alerts silently stop firing after DAG re-serialization

[adirhmonday]

Apache Airflow version

3.2.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

When a DAG is updated (any code change that produces a new serialized_dag row), DeadlineAlert records are not created for the new serialized_dag. All subsequent dag_runs for that DAG silently produce no Deadline entries and never fire deadline callbacks.

Root cause

In SerializedDagModel.write_dag() (airflow/models/serialized_dag.py), when the deadline definition hasn't changed between versions, _try_reuse_deadline_uuids() returns the existing UUIDs and the code sets:

# serialized_dag.py ~line 646-651
if deadline_uuid_mapping is not None:
    # All deadlines matched — reuse the UUIDs to preserve hash.
    # Clear the mapping since the alert rows already exist in the DB;
    # no need to delete and recreate identical records.
    dag.data["dag"]["deadline"] = existing_deadline_uuids
    deadline_uuid_mapping = {}

Then in the new serialized_dag creation path (when has_task_instances=True):

# line 734
cls._create_deadline_alert_records(new_serialized_dag, deadline_uuid_mapping)

Since deadline_uuid_mapping = {}, _create_deadline_alert_records returns immediately (if not uuid_mapping: return) — the new serialized_dag gets no deadline_alert rows.

When a dag_run is created, definitions/dag.py queries:

SELECT * FROM deadline_alert WHERE serialized_dag_id = <new_serialized_dag_id>

Finds nothing → no Deadline row inserted → triggerer never fires the callback.

The {} optimization is correct for the in-place UPDATE path (same serialized_dag row, existing deadline_alert records stay valid). It is wrong for the INSERT path (new row, no deadline_alert records exist for the new serialized_dag_id).

What you think should happen instead?

deadline_alert records should be created for every new serialized_dag row, regardless of whether the deadline definition changed.

Suggested fix

In _create_deadline_alert_records, when the new serialized_dag path is taken (INSERT, not UPDATE), generate new UUIDs and create new deadline_alert records even when the deadline definition matches the previous version. The empty-mapping optimization should only apply to the in-place UPDATE path (lines 691–713).

How to reproduce

1. Deploy a DAG with a DeadlineAlert — verify a deadline_alert record is created
2. Make any code change to the DAG file that produces a new dag_version (e.g. add a comment) without changing the deadline definition
3. Wait for git-sync to pick up the new version
4. Trigger a dag_run and wait past the deadline
5. No alert fires; the deadline table has no row for this run

Operating System

N/A

Versions of Apache Airflow Providers

N/A

Deployment

Other 3rd-party Helm chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[jasperjonkhans]

This appears to describe the same failure fixed by #68734, including the non-deadline edit and new serialized_dag insertion path.