Remove all_map_indices from task_state_store.clear() in context#68880
Merged
amoghrajesh merged 2 commits intoJun 23, 2026
Merged
Conversation
Lee-W
reviewed
Jun 23, 2026
Lee-W
approved these changes
Jun 23, 2026
Contributor
Author
|
Failures are unrelated, merging. |
cetingokhan
pushed a commit
to cetingokhan/airflow
that referenced
this pull request
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Was generative AI tooling used to co-author this PR?
What
task_state_store.clear(all_map_indices=True)was exposed on the context object available inside running tasks. Mapped tasks run concurrently, so any sibling that writes after the clear survives it and the result is unpredictable and impossible to reason about if you go down a debugging path as to why it's not cleared :). There is no practical use case for this inside a running task: the only candidate is of a designated "leader" index doing post-fleet cleanup which would require polling sibling state, which is not available on the context object.Current behaviour
A task could call
task_state_store.clear(all_map_indices=True)to attempt a fleet-wide wipe of state for all map indices of the task. Due to concurrent execution, the outcome could have a race condition and unreliable.Proposed change
Remove
all_map_indicesfromTaskStateStore.clear()in the task context, and also remove out the entire plumbing: the field onClearTaskStateStorecomms message, the supervisor forwarding, the client query param, and the execution API route query param.The flag is preserved at the backend and core API layer - external callers (UI, CLI) acting on a completed task group can still wipe all indices cleanly via the core API.
Alternative
If you need to clear state across all map indices of a mapped task group, do it from a downstream task using the CLI / core API which supports it.
Changes of Note
The removal touches the full stack from context to wire: context.py, comms.py, supervisor.py, client.py, and the execution API route. The Core API route and MetastoreStateBackend are intentionally untouched.
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.