Bug 1858763: static-pod-installer: remove deadlock by recreating installers that disappear

[sttts]

The line

ret.TargetRevision = currNodeState.CurrentRevision

led to a dead-lock because the installer controller only does anything if either

• all TargetRevisions are zero
• or the TargetRevision value is larger than CurrentRevision.

We went into the second case and the controller dead-locked.

Note: just before this code there is the ensureInstallerPod invocation. We run into the bug when the pod apply call fails (e.g. of some network issue).

Note: when the installer pod disappears again and again (for whatever reason), this could hotloop (of course we do the usual backoff in the controller though). But this is the same
for the existing OOM case.

[openshift-ci-robot]

@sttts: This pull request references Bugzilla bug 1858763, which is invalid:

• expected the bug to target the "4.6.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1858763: static-pod-installer: remove deadlock by recreating installers that disappear

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@sttts: This pull request references Bugzilla bug 1858763, which is invalid:

• expected the bug to target the "4.6.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1858763: static-pod-installer: remove deadlock by recreating installers that disappear

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sttts]

/bugzilla refresh

[openshift-ci-robot]

@sttts: This pull request references Bugzilla bug 1858763, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target release (4.6.0) matches configured target release for branch (4.6.0)
• bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mfojtik]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mfojtik, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sttts]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sttts]

/hold

until proof bump openshift/cluster-kube-apiserver-operator#939 shows some green.

[openshift-ci-robot]

@sttts: All pull requests linked via external trackers have merged:

• openshift/library-go#871

Bugzilla bug 1858763 has been moved to the MODIFIED state.

In response to this:

Bug 1858763: static-pod-installer: remove deadlock by recreating installers that disappear

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sttts]

/cherry-pick release-4.5

[sttts]

/cherry-pick release-4.4

[openshift-cherrypick-robot]

@sttts: new pull request created: #879

In response to this:

/cherry-pick release-4.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-cherrypick-robot]

@sttts: #871 failed to apply on top of branch "release-4.4":

Applying: static-pod-installer: recreate installers that disappeared
Using index info to reconstruct a base tree...
M	pkg/operator/staticpod/controller/installer/installer_controller.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/operator/staticpod/controller/installer/installer_controller.go
CONFLICT (content): Merge conflict in pkg/operator/staticpod/controller/installer/installer_controller.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 static-pod-installer: recreate installers that disappeared
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.