[Flaky] Garbage collector should not delete dependents that have both valid owner and owner that's waiting for dependents to be deleted

[aojea]

         there was a flake of this test today:

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/124361/pull-kubernetes-conformance-kind-ga-only-parallel/1780870571198779392

Originally posted by @neolit123 in > #120570 (comment)

it failed with

STEP: Found 1 events. - k8s.io/kubernetes/test/e2e/framework/debug/dump.go:46 @ 04/18/24 08:23:42.251
I0418 08:23:42.251282 68958 dump.go:53] At 2024-04-18 08:23:40 +0000 UTC - event for pod1: {kubelet kind-worker2} OutOfpods: Node didn't have enough resource: pods, requested: 1, used: 110, capacity: 110
I0418 08:23:42.259586 68958 resource.go:168] POD   NODE          PHASE   GRACE  CONDITIONS
I0418 08:23:42.259641 68958 resource.go:175] pod1  kind-worker2  Failed         []
IIRC there was one test that created a lot of pods

simpletest-rc-to-be-deleted-8r928

That is the test we need to fail the bug, as it is impacting the other tests

kubernetes/test/e2e/apimachinery/garbage_collector.go

Lines 714 to 719 in c016b0b

	framework.ConformanceIt("should not delete dependents that have both valid owner and owner that's waiting for dependents to be deleted", func(ctx context.Context) {
	clientSet := f.ClientSet
	rcClient := clientSet.CoreV1().ReplicationControllers(f.Namespace.Name)
	podClient := clientSet.CoreV1().Pods(f.Namespace.Name)
	rc1Name := "simpletest-rc-to-be-deleted"
	replicas := estimateMaximumPods(ctx, clientSet, 10, 100)

The test run a large number of Pods that may exhaust the capacity of the nodes and make other test to fail because are not able to schedule Pods

/sig api-machinery

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[aojea]

/kind flake

[alculquicondor]

Are we creating the default 12500 pods that would trigger the gc?

[aojea]

Are we creating the default 12500 pods that would trigger the gc?

kubernetes/test/e2e/apimachinery/garbage_collector.go

Lines 54 to 84 in c016b0b

	// estimateMaximumPods estimates how many pods the cluster can handle
	// with some wiggle room, to prevent pods being unable to schedule due
	// to max pod constraints.
	func estimateMaximumPods(ctx context.Context, c clientset.Interface, min, max int32) int32 {
	nodes, err := e2enode.GetReadySchedulableNodes(ctx, c)
	framework.ExpectNoError(err)
	availablePods := int32(0)
	for _, node := range nodes.Items {
	if q, ok := node.Status.Allocatable["pods"]; ok {
	if num, ok := q.AsInt64(); ok {
	availablePods += int32(num)
	continue
	}
	}
	// best guess per node, since default maxPerCore is 10 and most nodes have at least
	// one core.
	availablePods += 10
	}
	//avoid creating exactly max pods
	availablePods = int32(float32(availablePods) * 0.5)
	// bound the top and bottom
	if availablePods > max {
	availablePods = max
	}
	if availablePods < min {
	availablePods = min
	}
	return availablePods
	}