Automated cherry pick of #117865: Parallel StatefulSet pod create & delete

[aleksandra-malinowska]

Cherry pick of #117865 on release-1.25.

#117865: Refactor StatefulSet controller update logic

For details on the cherry pick process, see the cherry pick requests page.

Fixed a performance issue where pods weren't created/deleted in parallel for a StatefulSet with podManagementPolicy: Parallel.

[aleksandra-malinowska]

/kind bug
/priority important-soon
/assign @soltysh

[soltysh]

/lgtm
/approve

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: a959fb9294e5c5c7ef4398b0e8f5f1c79c3f8536

[soltysh]

/triage accepted

[Verolop]

/retest-required

[aleksandra-malinowska]

Looks like in 1.26, unit test fakes were improved to avoid sorting all pods to change one pod's state. New tests added in this PR use more pods and are simply too slow with 1.25 inefficient fakes (exceed per package limit of 180s).

I can either reduce the number of pods in test cases, or backport some of the test-only improvements from #112744. Either way, it won't be a clean cherry-pick, but the difference will be confined to unit tests.

@Verolop @soltysh if that's OK, I'd prefer to improve the fakes, so that we have the same test cases everywhere.

[aleksandra-malinowska]

Pushed a minimal subset of unit test changes from #112744 as a new commit. Let me know if you'd prefer me to squash it so every commit compiles.

[Verolop]

@aleksandra-malinowska it's ok if it's not a 'clean' cherry-pick
@soltysh can you please add your lgtm again?
thanks!

[soltysh]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aleksandra-malinowska, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/controller/statefulset/OWNERS [soltysh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 22d03541b6dc5b71ad983ef8eafddc69850424b2

[aleksandra-malinowska]

Unrelated flake likely due to #107414 (fix wasn't cherry-picked to 1.25):

=== FAIL: pkg/controller/volume/attachdetach/reconciler Test_Run_OneVolumeDetachFailNodeWithReadWriteOnce (1.13s)
W0712 15:04:41.258066   55960 mutation_detector.go:53] Mutation detector is enabled, this will result in memory leakage.
I0712 15:04:41.258485   55960 operation_generator.go:398] AttachVolume.Attach succeeded for volume "volume-name" (UniqueName: "fake-plugin/volume-name") from node "fail-detach-node" 
I0712 15:04:41.258753   55960 reconciler.go:346] "attacherDetacher.AttachVolume started" volume={VolumeToAttach:{MultiAttachErrorReported:false VolumeName:fake-plugin/volume-name VolumeSpec:0xc000013878 NodeName:fail-detach-node ScheduledPods:[&Pod{ObjectMeta:{pod-uid1  pod-uid1  pod-uid1  0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PodSpec{Volumes:[]Volume{},Containers:[]Container{},RestartPolicy:,TerminationGracePeriodSeconds:nil,ActiveDeadlineSeconds:nil,DNSPolicy:,NodeSelector:map[string]string{},ServiceAccountName:,DeprecatedServiceAccount:,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:nil,ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{},HostAliases:[]HostAlias{},PriorityClassName:,Priority:nil,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:nil,PreemptionPolicy:nil,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,OS:nil,HostUsers:nil,},Status:PodStatus{Phase:,Conditions:[]PodCondition{},Message:,Reason:,HostIP:,PodIP:,StartTime:<nil>,ContainerStatuses:[]ContainerStatus{},QOSClass:,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},},}]}}
E0712 15:04:42.285539   55960 reconciler.go:219] failed to get health of node fail-detach-node: node "fail-detach-node" not found
E0712 15:04:42.297667   55960 reconciler.go:219] failed to get health of node fail-detach-node: node "fail-detach-node" not found
E0712 15:04:42.308167   55960 reconciler.go:219] failed to get health of node fail-detach-node: node "fail-detach-node" not found
E0712 15:04:42.321998   55960 reconciler.go:219] failed to get health of node fail-detach-node: node "fail-detach-node" not found
E0712 15:04:42.332364   55960 reconciler.go:219] failed to get health of node fail-detach-node: node "fail-detach-node" not found
    reconciler_test.go:1605: Check volume <fake-plugin/volume-name> is reported as attached to node <fail-detach-node>, got false, expected true

/retest

[aleksandra-malinowska]

/retest-required

[Verolop]

/lgtm