Prevent container orphans when using apiserver.IsRunningWrapper

[xrstf]

What this PR does / why we need it:
When the IsRunningWrapper() function is fiddling with initContainers, we must make sure we only accept and deal with a known state. Otherwise it can happen that old init containers are left behind or copies are never cleaned up.

Which issue(s) this PR fixes:
Fixes #6328

Does this PR introduce a user-facing change?:

[ATTN] Fix orphaned apiserver-is-running initContainers in usercluster controlplane. This can cause a short reconciliation burst to bring older usercluster resources in all Seed clusters up to date. Tune the maxReconcileLimit if needed.

[xrstf]

/cherrypick release/v2.14
/cherrypick release/v2.15

[kubermatic-bot]

@xrstf: once the present PR merges, I will cherry-pick it on top of release/v2.14 in a new PR and assign it to you.

In response to this:

/cherrypick release/v2.14
/cherrypick release/v2.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[xrstf]

/retest

[kron4eg]

/retest
/lgtm
/approve

[kubermatic-bot]

LGTM label has been added.

Git tree hash: 833f68e310797d43d9a3a164b19d43b2100d2345

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kron4eg, xrstf

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kron4eg,xrstf]
• pkg/controller/OWNERS [kron4eg,xrstf]
• pkg/resources/OWNERS [kron4eg,xrstf]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xrstf]

/retest

[xrstf]

/retest

[kubermatic-triage-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

[kubermatic-bot]

@kubermatic-triage-bot:

In response to this:

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kubermatic-triage-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

[kubermatic-bot]

@kubermatic-triage-bot:

In response to this:

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kubermatic-triage-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

[kubermatic-bot]

@kubermatic-triage-bot:

In response to this:

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kubermatic-triage-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

[kubermatic-bot]

@kubermatic-triage-bot:

In response to this:

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kubermatic-triage-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

[kubermatic-bot]

@kubermatic-triage-bot:

In response to this:

/retest
This bot automatically retries jobs that failed/flaked on approved PRs

Review the full test history

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

Also, here is a cat.
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kron4eg]

/retest

[kubermatic-bot]

@xrstf: #6329 failed to apply on top of branch "release/v2.14":

Applying: explicitly set initContainers to orphans when using apiserver.IsRunningWrapper
Using index info to reconstruct a base tree...
A	pkg/controller/seed-controller-manager/openshift/resources/cloud_credential_operator.go
A	pkg/controller/seed-controller-manager/openshift/resources/console.go
A	pkg/controller/seed-controller-manager/openshift/resources/dns_operator.go
A	pkg/controller/seed-controller-manager/openshift/resources/kube_controller_manager.go
A	pkg/controller/seed-controller-manager/openshift/resources/kube_scheduler.go
A	pkg/controller/seed-controller-manager/openshift/resources/oauth.go
A	pkg/controller/seed-controller-manager/openshift/resources/openshift_apiserver_deployment.go
A	pkg/controller/seed-controller-manager/openshift/resources/openshift_controller_manager.go
A	pkg/controller/seed-controller-manager/openshift/resources/openshift_network_operator.go
A	pkg/controller/seed-controller-manager/openshift/resources/registry_operator.go
A	pkg/resources/clusterautoscaler/clusterautoscaler.go
A	pkg/resources/controllermanager/deployment.go
A	pkg/resources/gatekeeper/deployment.go
A	pkg/resources/kubernetes-dashboard/deployment.go
A	pkg/resources/kubestatemetrics/deployment.go
A	pkg/resources/machinecontroller/deployment.go
A	pkg/resources/machinecontroller/webhook.go
A	pkg/resources/metrics-server/deployment.go
A	pkg/resources/scheduler/deployment.go
A	pkg/resources/usercluster/deployment.go
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): pkg/resources/gatekeeper/deployment.go deleted in HEAD and modified in explicitly set initContainers to orphans when using apiserver.IsRunningWrapper. Version explicitly set initContainers to orphans when using apiserver.IsRunningWrapper of pkg/resources/gatekeeper/deployment.go left in tree.
Auto-merging api/pkg/resources/usercluster/deployment.go
Auto-merging api/pkg/resources/scheduler/deployment.go
Auto-merging api/pkg/resources/metrics-server/deployment.go
Auto-merging api/pkg/resources/machinecontroller/webhook.go
Auto-merging api/pkg/resources/machinecontroller/deployment.go
Auto-merging api/pkg/resources/kubestatemetrics/deployment.go
Auto-merging api/pkg/resources/kubernetes-dashboard/deployment.go
Auto-merging api/pkg/resources/controllermanager/deployment.go
Auto-merging api/pkg/resources/clusterautoscaler/clusterautoscaler.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/registry_operator.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/openshift_network_operator.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/openshift_controller_manager.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/openshift_apiserver_deployment.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/oauth.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/kube_scheduler.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/kube_controller_manager.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/dns_operator.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/console.go
Auto-merging api/pkg/controller/seed-controller-manager/openshift/resources/cloud_credential_operator.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 explicitly set initContainers to orphans when using apiserver.IsRunningWrapper
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherrypick release/v2.14
/cherrypick release/v2.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.