Bug 1857192: test/e2e: don't count central grpc-tls #899
Conversation
We expect a constant number of 5 secrets as per https://github.com/openshift/cluster-monitoring-operator/blob/f9567c5be644a867e41e2312f93c7183d5aa9c10/test/e2e/user_workload_monitoring_test.go#L894 However the underlying `countGRPCSecrets` method only lists secrets having the `monitoring.openshift.io/hash` label set which excludes `grpc-tls`. It is flaky because, during GRPC TLS rotation, there are indeed sometimes 5 secrets with the above hash label (pre- and post-rotation secrets). The fix is to expect 4 secrets only which is the what the rotation has to converge against.
openshift-ci-robot
added
the
approved
s-urbaniak
changed the title
openshift-ci-robot
added
bugzilla/severity-medium
|
@s-urbaniak: This pull request references Bugzilla bug 1857192, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker. 3 validation(s) were run on this bug
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
| // 4. openshift-user-workload-monitoring/thanos-ruler-grpc-tls-[hash] | ||
| // | ||
| // The central grpc-tls secret is verified independently by getting it directly | ||
| // and verifying if the force-rotation annotation has been removed. |
There was a problem hiding this comment.
Could we ensure that openshift-monitoring/grpc-tls is excluded from being counted?
There was a problem hiding this comment.
in fact we do that already (that was the actual flake cause) by setting the label selector in
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: paulfantom, s-urbaniak The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
@s-urbaniak: All pull requests linked via external trackers have merged: openshift/cluster-monitoring-operator#899. Bugzilla bug 1857192 has been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
We expect a constant number of 5 secrets
as per
cluster-monitoring-operator/test/e2e/user_workload_monitoring_test.go
Line 894 in f9567c5
However the underlying
countGRPCSecretsmethod only lists secretshaving the
monitoring.openshift.io/hashlabel setwhich excludes
grpc-tls.It is flaky because, during GRPC TLS rotation, there are indeed
sometimes 5 secrets with the above hash label (pre- and post-rotation secrets).
The fix is to expect 4 secrets only which is the what the rotation has to converge against.
/cc @openshift/openshift-team-monitoring