Skip to content

Allow string conversion to datetime for since_time in KPO logs#65498

Merged
jscheffl merged 3 commits intoapache:mainfrom
karenbraganz:string-to-datetime-kpo-logs
Apr 21, 2026
Merged

Allow string conversion to datetime for since_time in KPO logs#65498
jscheffl merged 3 commits intoapache:mainfrom
karenbraganz:string-to-datetime-kpo-logs

Conversation

@karenbraganz
Copy link
Copy Markdown
Collaborator

The since_seconds attribute in consume_logs (from class PodManager) and _write_logs (from class KubernetesPodOperator) is converted to None if since_time is not a datetime object. I have introduced code to check whether since_time is a string and convert it to a datetime object if possible instead of None.

This was suggested in PR #65232


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends KubernetesPodOperator/PodManager log fetching to handle since_time values provided as ISO-8601 strings by converting them into datetime objects instead of silently falling back to None.

Changes:

  • Convert string since_time values to datetimes in PodManager.fetch_container_logs() and KubernetesPodOperator._write_logs().
  • Expand unit test coverage for valid datetime, invalid input, and ISO-string since_time handling.
  • Refactor the PodManager since-time test to use time_machine rather than patching pendulum.now.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py Adds parsing of string since_time and broadens exception handling.
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/pod.py Adds parsing of string since_time in _write_logs and catches ValueError.
providers/cncf/kubernetes/tests/unit/cncf/kubernetes/utils/test_pod_manager.py Adds/updates tests for datetime, invalid, and string since_time scenarios.
providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py Adds a new _write_logs test for string since_time and adjusts invalid-case test wording.

Comment thread providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/pod.py Outdated
Comment thread providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py Outdated
Copy link
Copy Markdown
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks for the improvement and rework!

@jscheffl jscheffl merged commit 01c4c1d into apache:main Apr 21, 2026
111 checks passed
shahar1 added a commit to shahar1/airflow that referenced this pull request Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants