Fix TypeError clearing a DAG run when a queued-at deadline interval is stored as JSON - #70370
Open
hkc-8010 wants to merge 2 commits into
Open
Fix TypeError clearing a DAG run when a queued-at deadline interval is stored as JSON#70370hkc-8010 wants to merge 2 commits into
hkc-8010 wants to merge 2 commits into
Conversation
…s stored as JSON Since 3.3.0 the deadline_alert.interval column is JSON (migration 0117, apache#64751), holding a serialized timedelta or VariableInterval. _recalculate_dagrun_queued_at_deadlines still passed that raw dict into timedelta(seconds=...), so clearing a DAG run or task instance whose DAG has a DagRunQueuedAtDeadline failed with "TypeError: unsupported type for timedelta seconds component: dict" (HTTP 500). Decode the interval the same way DagRun creation does (decode_deadline_alert), then resolve a VariableInterval to a timedelta before recalculating. The existing test stored the interval as a plain float and never exercised the JSON path; it now stores the serialized envelope and is parametrized over fixed-timedelta and VariableInterval. closes: apache#70368
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Clearing a DAG run or task instance whose DAG defines a deadline referencing the run's queued-at time (
DeadlineReference.DAGRUN_QUEUED_AT) raisedTypeError: unsupported type for timedelta seconds component: dictand surfaced as an HTTP 500 in the API server.Since 3.3.0 the
deadline_alert.intervalcolumn is JSON (migration0117_3_3_0_change_deadline_interval_to_json.py, from #64751), holding a serializedtimedeltaorVariableInterval, e.g.{"__classname__": "datetime.timedelta", "__version__": 2, "__data__": 3600.0}._recalculate_dagrun_queued_at_deadlines(called fromclear_task_instances) still passed that raw dict intotimedelta(seconds=...). Every other consumer (DagRun creation inserialization/definitions/dag.py, the model__repr__) decodes the JSON first; only the clear path was missed.The fix decodes the interval the same way DagRun creation does, via
decode_deadline_alert, then resolves aVariableIntervalto atimedeltabefore recalculating.Changes
airflow-core/src/airflow/models/taskinstance.py: decode the JSON interval viadecode_deadline_alertand resolveVariableIntervalin_recalculate_dagrun_queued_at_deadlines.airflow-core/tests/unit/models/test_taskinstance.py: the existing test stored the interval as a plain float, so it never exercised the JSON path. It now stores the serialized envelope and is parametrized over a fixedtimedeltaand aVariableInterval. Both cases fail onmainwith the reportedTypeErrorand pass with the fix.closes: #70368
PR Checklist
mainbranchmain)prekhooks pass on the changed files (ruff, ruff-format, mypy for airflow-core)