Fix issue with parsing error logs in the KPO #15638
Conversation
boring-cyborg
bot
added
provider:cncf-kubernetes
dimberman
force-pushed
the
fix-timestamp-kpo-logging
branch
from
79e2d5a
to
e9aa0e8
Compare
|
The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason. |
| @@ -169,7 +172,7 @@ def monitor_pod(self, pod: V1Pod, get_logs: bool) -> Tuple[State, Optional[str]] | |||
| time.sleep(2) | |||
| return self._task_status(self.read_pod(pod)), result | |||
|
|
|||
| def parse_log_line(self, line: str) -> Tuple[str, str]: | |||
| def parse_log_line(self, line: str) -> Tuple[Optional[Union[Date, Time, DateTime, Duration]], str]: | |||
There was a problem hiding this comment.
This type is overly broad - it should be Optional[Date time] shouldn't it?
|
@dimberman Just checking if this is still relevant and important to finish up? |
|
@vikramkoka this ticket is still relevant but unfortunately I don't have time to address it any time soon :( |
|
@vikramkoka this is important effecting quite a few of our jobs (~25%) now we have moved to 2.1.0, |
This fixes an issue where logs that do not have timestamps cause the
KubernetesPodOperator to crash. Basically error logs created by airflow
do not have timestamps, which was causing an unhandled exception that
would kill the task. This PR handles that exception and ensures
continued task processing
ashb
force-pushed
the
fix-timestamp-kpo-logging
branch
from
e9aa0e8
to
bb9a040
Compare
|
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
github-actions
bot
added
the
okay to merge
This fixes an issue where logs that do not have timestamps cause the
KubernetesPodOperator to crash. Basically error logs created by airflow
do not have timestamps, which was causing an unhandled exception that
would kill the task. This PR handles that exception and ensures
continued task processing
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.