Skip to content

Only log task state cleanup for configured worker state store backends#68878

Merged
vatsrahul1001 merged 2 commits into
apache:mainfrom
astronomer:fix-wrong-log-aip-103
Jun 23, 2026
Merged

Only log task state cleanup for configured worker state store backends#68878
vatsrahul1001 merged 2 commits into
apache:mainfrom
astronomer:fix-wrong-log-aip-103

Conversation

@amoghrajesh

Copy link
Copy Markdown
Contributor

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Noticed this while testing https://cwiki.apache.org/confluence/display/AIRFLOW/Test+cases+AIP103+Task+State+Management with beta1 for 3.3.0, we should only ever be logging Clearing task state from custom backend as clear_on_success is enabled. The database references will be cleared by the API server. when a custom backend is configured AND not when clear_on_success is set on a default backend which could be really confusing when debugging things.


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@phanikumv phanikumv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing a CI failure. Replace this line with the below code

with mock.patch(
"airflow.sdk.execution_time.context._get_worker_state_store_backend",return_value=mock_backend),
mock.patch("airflow.sdk.execution_time.task_runner._get_worker_state_store_backend", return_value=mock_backend
):

@Lee-W Lee-W left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add a small unit test for it

@amoghrajesh

Copy link
Copy Markdown
Contributor Author

@Lee-W @phanikumv handled your comments, ptal!

@vatsrahul1001 vatsrahul1001 merged commit a0d7ae6 into apache:main Jun 23, 2026
102 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in AIP-103: Task State Management Jun 23, 2026
@vatsrahul1001 vatsrahul1001 deleted the fix-wrong-log-aip-103 branch June 23, 2026 08:00
cetingokhan pushed a commit to cetingokhan/airflow that referenced this pull request Jun 24, 2026
apache#68878)

* Only log task state cleanup for configured worker state store backends

* fixing CI and adding test for wei
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

5 participants