Skip to content

Add retries for _write_logs method in KubernetesPodOperator#64471

Open
wolfdn wants to merge 3 commits intoapache:mainfrom
boschglobal:feature/add-retries-for-write-logs-method
Open

Add retries for _write_logs method in KubernetesPodOperator#64471
wolfdn wants to merge 3 commits intoapache:mainfrom
boschglobal:feature/add-retries-for-write-logs-method

Conversation

@wolfdn
Copy link
Copy Markdown
Contributor

@wolfdn wolfdn commented Mar 30, 2026

Description

This PR adds retries using Tenacity to the _write_logs method in KubernetesPodOperator.
Previously, this method was logging Reading of logs interrupted with error ...; will retry. when it encountered an ApiException or HTTPError - but actually it did not retry (maybe the retry logic went missing during some refactoring?).

Apart from adding retries, the behaviour is kept the same: If all retries fail, KubernetesPodOperator will print another warning but then proceed with pod cleanup.

Other changes

  • Replace from airflow.utils import timezone with from airflow.sdk import timezone - this was consistently failing for me in pre-push hooks otherwise, I assume this is needed now
  • Fix order of mocks in test_async_write_logs_handler_api_exception test

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    GitHub Copilot - Claude Opus 4.6

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg bot added area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels Mar 30, 2026
@jscheffl
Copy link
Copy Markdown
Contributor

@wolfdn Backcompat tests are failing, can you check these?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants