pkg/kubelet/pleg: do not include exited and dead pods in running_pod_count metric

[paulfantom]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Stop including exited and dead pods in the kubelet_running_pod_count metric.

Which issue(s) this PR fixes:

Fixes #92180

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Kubelet metric `kubelet_running_pod_count` does not include exited pods.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

Welcome @paulfantom!

It looks like this is your first PR to kubernetes/kubernetes 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/kubernetes has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @paulfantom. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lilic]

/ok-to-test

[lilic]

cc @kubernetes/sig-instrumentation-pr-reviews

[brancz]

Looks good from instrumentation side.

/lgtm

Sig node needs to approve.

cc @kubernetes/sig-node-bugs

[zhouya0]

You should also correct test case, and by the way, g.podRecords.setCurrent(pods) seems uses all pods before maybe we should not change this. Just change updateRunningPodAndContainerMetrics(pods) .

[derekwaynecarr]

This change is not correct as it will break the ability to properly computeEvents in the pleg.

Please update as requested.

/hold

[derekwaynecarr]

This change has bad side-effects. It is intentional that all pods are fetched in this context so we can properly build the pleg. The change should be that updateRunningPodAndContainerMetrics should do the desired filtering.

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: paulfantom
To complete the pull request process, please assign sjenning
You can assign the PR to them by writing /assign @sjenning in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/kubelet/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[paulfantom]

I changed it to base kubelet_running_pod_count only on pods in Running phase.

[k8s-ci-robot]

@paulfantom: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kubernetes-bazel-test	9052521	link	/test pull-kubernetes-bazel-test
pull-kubernetes-e2e-kind-ipv6	9052521	link	/test pull-kubernetes-e2e-kind-ipv6

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[SergeyKanzhelev]

Will this method ToAPIPod update the Status? I cannot find where it does it:

kubernetes/pkg/kubelet/container/runtime.go

Lines 598 to 613 in 5ed7b1a

	// ToAPIPod converts Pod to v1.Pod. Note that if a field in v1.Pod has no
	// corresponding field in Pod, the field would not be populated.
	func (p *Pod) ToAPIPod() *v1.Pod {
	var pod v1.Pod
	pod.UID = p.ID
	pod.Name = p.Name
	pod.Namespace = p.Namespace
	for _, c := range p.Containers {
	var container v1.Container
	container.Name = c.Name
	container.Image = c.Image
	pod.Spec.Containers = append(pod.Spec.Containers, container)
	}
	return &pod
	}

[paulfantom]

I was hoping it did, but apparently it doesn't which can be seen in failing bazel-test. Any ideas how to get pod status/phase without iterating over all containers and thus duplicating logic?

[paulfantom]

Closing in favor of #85983