Allow argument-provision checks of templated fields in operator __init__ - #70505
Open
shahar1 wants to merge 1 commit into
Open
Allow argument-provision checks of templated fields in operator __init__#70505shahar1 wants to merge 1 commit into
shahar1 wants to merge 1 commit into
Conversation
shahar1
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
potiuk
as code owners
July 27, 2026 08:42
Which arguments a Dag author passed is a question only the constructor can answer: with render_template_as_native_obj=True a provided field can render to None, so the same check in execute() reports a supplied argument as missing. Forcing those checks out of __init__ also turns a static authoring mistake into a per-task-instance runtime failure and, because a rendered value can legitimately be falsy, invites truthiness tests that silently accept conflicting arguments. Assignments are now walked recursively so a branch cannot hide a derived write to a template field behind a provision guard.
shahar1
force-pushed
the
validate-init-provision-checks
branch
from
July 27, 2026 08:58
adfa421 to
0026dec
Compare
This was referenced Jul 27, 2026
This was referenced Jul 27, 2026
1 task
mitre88
added a commit
to mitre88/airflow
that referenced
this pull request
Jul 27, 2026
The provision checks in GCSDeleteObjectsOperator are false positives under the relaxed validator definition proposed in apache#70505, so they stay in the constructor and the class keeps its exemption entry.
This was referenced Jul 27, 2026
vincbeck
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related: #70296, #70347, #70297
Human Summary
As it turns out, the check I've introduced in #70297 might have been too strict, and we should allow some specific checks to still run in the constructor. This PR relaxes the requirements and removes false positives (according to the new definition) from the exemptions list.
Created a separate issue (#70503) to track reverts.
AI Summary
Click here
validate-operators-initflags every read of a template-field parameter in__init__. That is right for reads of the value, but wrong for reads of provision — "did the author pass this argument?" — which is what mutually-exclusive-argument checks ask.The constructor is in fact the only place that can answer it. With
render_template_as_native_obj=Truea provided field renders toNone, so the same check inexecute()reports a supplied argument as missing. Forcing the move also turns a static authoring mistake into a per-task-instance runtime failure, and — because a rendered value can legitimately be falsy — invites truthiness tests that silently accept conflicting arguments.This adds one sanctioned pattern:
field is None/field is not Noneon a template-field parameter (orself.<field>) is allowed in__init__. Everything else — truthiness,in,.startswith,isinstance, transformations — stays flagged.Measured against the burn-down. Replaying both checkers over every exemption entry ever removed (31 PRs, 48 entries): 8 entries across 5 PRs would have needed no operator code change at all — #70341, #70338, #70348, #70392, #70326 moved pure
is Noneexclusivity checks intoexecute()verbatim. Two cost more than churn: #70348 broke the Rendered Templates view and needed follow-up #70373, and #70326 made a released operator argument required. Five currently-listed entries clear with no code change and are removed here; two of them are already claimed by contributors on #70296.The remaining 40 entries stay flagged, correctly — they are genuine value reads (
ruleset.strip(),isinstance(mongo_query, list),source not in SUPPORTED_SOURCES).Second half of the change: template-field assignments are now walked recursively. Sanctioning the
is Noneread would otherwise letif field is None: self.field = derive(other)through, since the assignment check only looked at top-level statements. That also closes a pre-existing hole with an unrelated guard.Testing Done
scripts/tests/ci/prek/test_validate_operators_init.py: 35 passed. 8 of the 10 new/changed cases fail against the pre-change checker; the 2 that don't are guards against over-sanctioning (truthiness andfoo.get('x') is Nonemust stay flagged).files:regex: exit 0, no stale exemptions.prek run --from-ref main --stage pre-commit: clean (includes mypy for scripts).Reviewer note: reconciling merged precedent
Five merged burn-down PRs moved pure provision checks into
execute()and left comments and regression tests asserting that placement. Nothing breaks — the new rule permitsis Nonein__init__, it does not require it — but those files now demonstrate the pattern the docs discourage, and contributors on #70296 learn the rule by reading them. Reverting them is tracked separately so this PR stays reviewable.No newsfragment: prek hook, contributing docs, and a clarification to an existing limitations section. Not user-facing behaviour.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines