[community-4.8] WINC-571: [wmco] Prevent OLM upgrades when operator is busy #760
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/lgtm |
This commit introduces a new time limit to wait for changes upon to resources to take effect. This applies to resource creation, deletion & updates/patches. Patching the operator's Upgradeable condition can take longer to so a longer retry timeout is chosen rather than erroring out too quickly. (cherry picked from commit 1dc44ab)
This commit introduces constants that denote the name of each controller. These names are used in log messages, events, and other outputs. (cherry picked from commit 2c1f115)
This commit introduces the reporting of operator conditions back to OLM. Specifically, the `Upgradeable` condition is set to `False` when WMCO is processing any child components (Windows nodes, private key secret, CSRs), blocking OLM from replacing the operator CSV during this time. The `Upgradeable` condition is set back to `True` when no processing is currently ongoing, meaning that all controllers: - reconciled without error - did not request a requeue If there is at least one busy controller, operator cannot be upgraded (Upgradeable condition set to False), preventing unsafe updates. A mutex lock is maintained to secure access to the busy controllers set and the patching of the OperatorCondition resource. This ensures smooth transitions from one EUS version of OpenShift to another. A warning event is sent off when there is a failure getting or patching the OperatorCodnition resource. The event serves to alert users, as such failures can block WMCO controllers from executing reconcile steps (as recommended by OLM). A recorder was added to the secret reconciler struct to bring it in line with the other reconcilers. (cherry picked from commit 3fa7419)
This commit tests the operator's ability to communicate to OLM that it is currently not interruptable for upgrade. Checks are added to ensure the oeprator's Upgradeable condition is False when it is processing any sub-components, and reset to True when it is not reconciling/safe for upgrade. (cherry picked from commit f57effd)
Ran: - `go get "github.com/operator-framework/api/pkg/operators/v2"` - `go mod vendor && go mod tidy`
saifshaikh48
force-pushed
the
prevent-concurrent-upgrade-c4.8
branch
from
65159ab
to
f28efe3
Compare
|
/lgtm |
|
@saifshaikh48: you cannot LGTM your own PR. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: aravindhp, saifshaikh48 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Manual backport of #618 due to vendoring conflict.