Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flake when SCCs are not associated yet #26225

Closed

Conversation

ravisantoshgudimetla
Copy link
Contributor

https://bugzilla.redhat.com/show_bug.cgi?id=1961204 was punted
to 4.9 release, however occasionally we see multus not
getting SCCs in time causing auth issues. This commit
causes the test to flake in such scenarios instead of
failing. This is a temporary measure to get better
signal for 4.8 upgrades. We need to remove this
when the associated bug has been fixed

cc @vrutkovs @dcbw

@openshift-ci openshift-ci bot requested review from mfojtik and spadgett June 14, 2021 14:37
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 14, 2021

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ravisantoshgudimetla
To complete the pull request process, please assign bparees after the PR has been reviewed.
You can assign the PR to them by writing /assign @bparees in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@@ -42,6 +42,10 @@ func testPodSandboxCreation(events monitorapi.Intervals) []*ginkgo.JUnitTestCase
flakes = append(flakes, fmt.Sprintf("%v - multus is unable to get pods due to LB disruption https://bugzilla.redhat.com/show_bug.cgi?id=1927264 - %v", event.Locator, event.Message))
continue
}
if strings.Contains(event.Message, "Multus") && strings.Contains(event.Message, "error getting pod") && strings.Contains(event.Message, "is forbidden: User \"system:serviceaccount:openshift-multus:multus\"") {
flakes = append(flakes, fmt.Sprintf("%v - multus is unable to get pods due to not getting required SCCs https://bugzilla.redhat.com/show_bug.cgi?id=1961204 - %v", event.Locator, event.Message))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a GET request doesn't ever use SCC

@deads2k
Copy link
Contributor

deads2k commented Jun 14, 2021

the fix and bug dont' appear related

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 14, 2021
https://bugzilla.redhat.com/show_bug.cgi?id=1961204 was punted
to 4.9 release, however occasionally we see multus not
getting SCCs in time causing auth issues. This commit
causes the test to flake in such scenarios instead of
failing. This is a temporary measure to get better
signal for 4.8 upgrades. We need to remove this
when the associated bug has been fixed
@deads2k
Copy link
Contributor

deads2k commented Jun 15, 2021

one possibility is that there is no ready kube-apiserver. Another possibility is that there is a ready kube-apiserver, but for some reason the LB isn't detecting it.

openshift/kubernetes#807 should show the reality of being ready at any given time. I'm not sure how to collect the list of endpoints for a load balancer. Perhaps the SPLAT team knows how to do this?

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 19, 2021

@ravisantoshgudimetla: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 19, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 25, 2021

@ravisantoshgudimetla: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-gcp-csi 595740d link /test e2e-gcp-csi
ci/prow/e2e-aws-csi 595740d link /test e2e-aws-csi
ci/prow/e2e-agnostic-cmd 595740d link /test e2e-agnostic-cmd
ci/prow/e2e-gcp 595740d link /test e2e-gcp
ci/prow/e2e-aws-disruptive 595740d link /test e2e-aws-disruptive
ci/prow/e2e-gcp-upgrade 595740d link /test e2e-gcp-upgrade
ci/prow/e2e-gcp-disruptive 595740d link /test e2e-gcp-disruptive
ci/prow/e2e-aws-jenkins 595740d link /test e2e-aws-jenkins

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 23, 2021
@openshift-bot
Copy link
Contributor

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Oct 23, 2021
@openshift-bot
Copy link
Contributor

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Nov 22, 2021

@openshift-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot closed this Nov 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants