Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Continue streaming kubelet logs when runtime is unavailable #124025

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

saschagrunert
Copy link
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

Container runtimes are able to run existing containers even when their main CRI server is not available for any reason. The call to the container status RPC happens quite frequently during log parsing, means that a single CRI interruption will also abort streaming the logs.

We now check that specific use case and continue following the log streaming if the CRI is unavailable. We still abort the streaming accordingly if the CRI comes back and the container status reports that the workload has exited.

Which issue(s) this PR fixes:

Refers to cri-o/cri-o#7826

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Continue streaming kubelet logs when the CRI server of the runtime is unavailable.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

None

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Mar 22, 2024
@saschagrunert
Copy link
Member Author

PTAL @kubernetes/sig-node-pr-reviews

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Mar 22, 2024
@bart0sh bart0sh added this to Triage in SIG Node PR Triage Mar 22, 2024
@kannon92
Copy link
Contributor

Is this patch just related to kubelet logs?

@kannon92
Copy link
Contributor

Is this patch just related to kubelet logs?

I researched this. The code that @saschagrunert added is only used in https://github.com/saschagrunert/kubernetes/blob/7ea3d0245a63fbbba698f1cb939831fe8143db3e/pkg/kubelet/kuberuntime/logs/logs.go#L442

So this change is actually only limited to logs.

@bart0sh
Copy link
Contributor

bart0sh commented Mar 24, 2024

/triage accepted
/priority important-longterm

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Mar 24, 2024
@bart0sh bart0sh moved this from Triage to Needs Reviewer in SIG Node PR Triage Mar 24, 2024
@bart0sh bart0sh added this to Triage in SIG Node CI/Test Board Mar 24, 2024
@haircommander
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 25, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 6224fb14640c25fc82923a9615cdac751e487253

@bart0sh bart0sh moved this from Needs Reviewer to Needs Approver in SIG Node PR Triage Mar 25, 2024
@bart0sh bart0sh moved this from Triage to Archive-it in SIG Node CI/Test Board Mar 31, 2024
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 1, 2024
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 3, 2024
@k8s-ci-robot
Copy link
Contributor

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 3, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 3, 2024
@saschagrunert
Copy link
Member Author

@kubernetes/sig-node-pr-reviews PTAL

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 3, 2024
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 11, 2024
Container runtimes are able to run existing containers even when their
main CRI server is not available for any reason. The call to the
container status RPC happens quite frequently during log parsing, means
that a single CRI interruption will also abort streaming the logs.

We now check that specific use case and continue following the log
streaming if the CRI is unavailable. We still abort the streaming
accordingly if the CRI comes back and the container status reports that
the workload has exited.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jun 11, 2024
@saschagrunert
Copy link
Member Author

Rebased, @kubernetes/sig-node-pr-reviews PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Status: Needs Approver
SIG Node PR Triage
Needs Approver
Development

Successfully merging this pull request may close these issues.

None yet

5 participants