Extend CVO alerts to cover update retrieval #357
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openshift-ci-robot
added
the
do-not-merge/work-in-progress
openshift-ci-robot
added
the
approved
jottofar
force-pushed
the
ota-151-extend-alerts
branch
from
a80d7eb
to
ca9a94a
Compare
|
/test e2e-aws |
jottofar
force-pushed
the
ota-151-extend-alerts
branch
2 times, most recently
from
5010537
to
2d99330
Compare
|
/test e2e-aws |
jottofar
force-pushed
the
ota-151-extend-alerts
branch
2 times, most recently
from
1487e3b
to
a9f575a
Compare
wking
reviewed
Apr 23, 2020
install/0000_90_cluster-version-operator_02_servicemonitor.yaml
Outdated
Show resolved
Hide resolved
wking
reviewed
Apr 23, 2020
install/0000_90_cluster-version-operator_02_servicemonitor.yaml
Outdated
Show resolved
Hide resolved
wking
reviewed
Apr 23, 2020
wking
reviewed
Apr 23, 2020
jottofar
force-pushed
the
ota-151-extend-alerts
branch
from
a9f575a
to
31b703e
Compare
jottofar
force-pushed
the
ota-151-extend-alerts
branch
2 times, most recently
from
1c9b7dd
to
6aaa85a
Compare
jottofar
changed the title
openshift-ci-robot
removed
the
do-not-merge/work-in-progress
|
/retest |
1 similar comment
|
/retest |
|
/test e2e-aws |
wking
added a commit
to wking/openshift-release
that referenced
this pull request
Apr 30, 2020
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786
|
/test e2e-aws |
wking
reviewed
May 4, 2020
install/0000_90_cluster-version-operator_02_servicemonitor.yaml
Outdated
Show resolved
Hide resolved
wking
reviewed
May 4, 2020
jottofar
force-pushed
the
ota-151-extend-alerts
branch
8 times, most recently
from
b81e6f0
to
3cc123d
Compare
|
/test images |
|
/retest |
Added metric cluster_version_operator_update_retrieval_timestamp_seconds to track last known successful update retrieval time and use metric cluster_operator_conditions to determine update retrieval failure reason. Added alert CannotRetrieveUpdates that fires when metric cluster_version_operator_update_retrieval_timestamp_seconds is >= 3600 unless reason, as reported by cluster_operator_conditions is NoChannel. Alert reports last known successful update retrieval time, reason unable to retrieve updates, and console URL to get more information.
jottofar
force-pushed
the
ota-151-extend-alerts
branch
from
3cc123d
to
1005398
Compare
|
/test e2e-aws-upgrade |
1 similar comment
|
/test e2e-aws-upgrade |
|
🎉 /lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jottofar, wking The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/test e2e-aws-upgrade |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
2 similar comments
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/test e2e-aws-upgrade |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
|
/retest Please review the full test history for this PR and help us cut down flakes. |
stbenjam
added a commit
to stbenjam/dev-scripts
that referenced
this pull request
Jun 3, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786
stbenjam
added a commit
to stbenjam/dev-scripts
that referenced
this pull request
Jun 3, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. See this related PR [6] which does this for other platforms IPI install step. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786 [6]: openshift/release#8631
stbenjam
added a commit
to stbenjam/dev-scripts
that referenced
this pull request
Jun 3, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. See this related PR [6] which does this for other platforms IPI install step. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786 [6]: openshift/release#8631 Co-authored-by: W. Trevor King <wking@tremily.us>
openshift-merge-robot
pushed a commit
to openshift-metal3/dev-scripts
that referenced
this pull request
Sep 23, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. See this related PR [6] which does this for other platforms IPI install step. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786 [6]: openshift/release#8631 Co-authored-by: W. Trevor King <wking@tremily.us> Co-authored-by: W. Trevor King <wking@tremily.us>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.