Automated cherry pick of #112650: kubelet: increase log level for Path does not exist message

[rphillips]

Cherry pick of #112650 on release-1.25.

#112650: kubelet: increase log level for Path does not exist message

For details on the cherry pick process, see the cherry pick requests page.

[k8s-ci-robot]

@rphillips: This cherry pick PR is for a release branch and has not yet been approved by Release Managers.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick, it must first be approved (/lgtm + /approve) by the relevant OWNERS.

AFTER it has been approved by code owners, please leave the following comment on a line by itself, with no leading whitespace: /cc kubernetes/release-managers

(This command will request a cherry pick review from Release Managers and should work for all GitHub users, whether they are members of the Kubernetes GitHub organization or not.)

For details on the patch release process and schedule, see the Patch Releases page.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@rphillips: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rphillips
Once this PR has been reviewed and has the lgtm label, please assign sjenning for approval by writing /assign @sjenning in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/kubelet/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rphillips]

/king bug
/priority important-soon
/sig node

[SergeyKanzhelev]

Sorry, I missed and didn't have a chance to comment on the original PR. The original bug suggests that there are so many these logs that the kubelet grows out of memory. Is hiding the message solves the problem? Will the function keep being called over and over?

/hold

[rphillips]

@SergeyKanzhelev The issue mentions a memory leak separate from the log spam. I am unsure about the memory leak, but the leak is probably not in the logger. I don't think we should hold this PR because it clearly fixes a log line that shouldn't be in the normal run path.

[SergeyKanzhelev]

@SergeyKanzhelev The issue mentions a memory leak separate from the log spam. I am unsure about the memory leak, but the leak is probably not in the logger. I don't think we should hold this PR because it clearly fixes a log line that shouldn't be in the normal run path.

Is there understanding why this line was printed so many times that the issue complained about the log spam? Is this code suppose to be called so many time?

[rphillips]

The function is called by HandlePodCleanups via the Kubelet's HouseKeeping timer (IIRC every 10s). So the kubelet is trying to cleanup some pod paths, getting into a situation where the path has already been cleaned up, and then printing the log spam message. This is for every pod or orphaned pod on the node :).

[SergeyKanzhelev]

The function is called by HandlePodCleanups via the Kubelet's HouseKeeping timer (IIRC every 10s). So the kubelet is trying to cleanup some pod paths, getting into a situation where the path has already been cleaned up, and then printing the log spam message. This is for every pod or orphaned pod on the node :).

is it once per pod or constantly? Will this ever end or it will continue trying to clean up till kubelet restarted? (I'm only speculating based on the original issue report). The problem with hiding a log message in a never-ending retries is that we are hiding the ticking bomb or a memory leak and CPU waste.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[marquiz]

ping @rphillips is this still valid?

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.