Fix drifting data intervals for monthly/yearly schedules with catchup disabled#69143
Merged
Lee-W merged 5 commits intoJun 30, 2026
Merged
Conversation
… disabled When a Dag scheduled with relativedelta(months=...) or relativedelta(years=...) runs with catchup=False, the first data interval after the Dag is unpaused was snapped to a fixed 30-day (or 365-day) grid measured from the Unix epoch. Real calendar months and years vary in length, so the resulting interval never aligned to the Dag's start_date and its day-of-month walked backwards over time. The logical date a monthly Dag received therefore depended on when it happened to be unpaused rather than being a stable function of its schedule.
uranusjr
reviewed
Jun 29, 2026
shahar1
added
backport-to-v3-3-test
Move the calendar-delta stepping next to the fixed-delta rounding so both schedule kinds resolve their boundary through a single helper, leaving _skip_to_latest a single expression. No behavior change.
uranusjr
reviewed
Jun 30, 2026
uranusjr
reviewed
Jun 30, 2026
uranusjr
approved these changes
Jun 30, 2026
Lee-W
reviewed
Jun 30, 2026
Contributor
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human Summary
We encountered this issue in my work when we scheduled a Dag on a past
start_datewithcatchup=Falseand schedule ofrelativedelta(months=1). In the specific case, the data interval of the first run drifted in about 5 days that it should be (e..g,2026-05-01became2025-05-06). The problem was that in these case it calculates the date over multiplications of fixed 30 days since the Unix epoch (1970-01-01), which naturally leads to a drift due to different number of days in a month since then (there's an equivalent issue for using years due to leap years). This PR should fix it.AI Summary
Click here
Monthly and yearly schedules (`relativedelta(months=...)` / `relativedelta(years=...)`) produced mis-aligned, drifting data intervals when `catchup=False`.DeltaDataIntervalTimetable._skip_to_latest(only reached on thecatchup=Falsepath) rounded "now" to a fixed 30-day / 365-day grid anchored at the Unix epoch. Calendar months and years are not constant in length, so the first data interval after a Dag is unpaused never aligned to the Dag'sstart_date, and the chosen day-of-month walked backwards over time (e.g.06-06 → 07-06 → 08-05 → 09-04). The logical date a monthly Dag received therefore depended on when it happened to be unpaused rather than being a stable function of its schedule.The fix anchors calendar-based deltas on
start_dateand advances one period at a time, so the boundaries match whatcatchup=Trueproduces (includingrelativedelta's path-dependent day-clamping, e.g. Jan 31 → Feb 28 → Mar 28).timedeltaand fixed-unitrelativedeltaschedules are untouched — they keep the existing O(1) epoch rounding, which is correct for fixed deltas.Added unit tests for monthly/yearly alignment, the no-
start_datecase, and day-clamping; all fail on the previous code and pass with the fix. The existing hourlytimedelta/relativedeltaandcatchup=Truetests guard the unchanged path.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines