Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Revert "Bug 2000216: Image policy should mutate DeploymentConfigs, StatefulSets, and new CronJobs" #1032

Closed

Conversation

stbenjam
Copy link
Member

@stbenjam stbenjam commented Nov 3, 2021

Reverts #1014

Starting with 4.10.0-0.nightly-2021-11-03-064540, we are failing release payload acceptance on SDN upgrades. Out of all the PR's this looks like possibly the most relevant. Reverting to test the hunch.

I'm not entirely clear what's wrong, but upgrades are failing with the \"master\" pool should be updated before the CVO reports available at the new version and authentication operator failing to upgrade.

Example failure: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.10-e2e-aws-upgrade/1455719487931158528

The timeline I can piece together from the above looks like this:

@ 03:36:09, etcd reported unhealthy members: ip-10-0-169-115.us-west-1.compute.internal , which is a master

@ 03:36:27 Node ip-10-0-169-115.us-west-1.compute.internal status is now: NodeSchedulable

@ 03:36:33 Status for clusteroperator/openshift-apiserver changed: Degraded message changed from "APIServerDeploymentDegraded: 1 of 3 requested instances are unavailable for apiserver.openshift-apiserver

@ 03:36:33 Created container oauth-apiserver on the ip-10-0-169-115 master

@ 03:36:36 master was updating after cluster version reached level: the "master" pool should be updated before the CVO reports available at the new version

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 3, 2021
@openshift-ci-robot openshift-ci-robot added the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Nov 3, 2021
@openshift-ci-robot
Copy link

@stbenjam: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

@openshift-ci openshift-ci bot added the bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. label Nov 3, 2021
@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

@stbenjam: This pull request references Bugzilla bug 2000216, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.10.0) matches configured target release for branch (4.10.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @xiuwang

In response to this:

[WIP] Revert "Bug 2000216: Image policy should mutate DeploymentConfigs, StatefulSets, and new CronJobs"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Nov 3, 2021
@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

@stbenjam: An error was encountered querying GitHub for users with public email (xiuwang@redhat.com) for bug 2000216 on the Bugzilla server at https://bugzilla.redhat.com. No known errors were detected, please see the full error message for details.

Full error message. non-200 OK status code: 403 Forbidden body: "{\n \"documentation_url\": \"https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits\",\n \"message\": \"You have exceeded a secondary rate limit. Please wait a few minutes before you try again.\"\n}\n"

Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

[WIP] Revert "Bug 2000216: Image policy should mutate DeploymentConfigs, StatefulSets, and new CronJobs"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot requested a review from xiuwang November 3, 2021 12:14
@stbenjam
Copy link
Member Author

stbenjam commented Nov 3, 2021

/test e2e-aws-upgrade

@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

@stbenjam: This pull request references Bugzilla bug 2000216, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.10.0) matches configured target release for branch (4.10.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @xiuwang

In response to this:

[WIP] Revert "Bug 2000216: Image policy should mutate DeploymentConfigs, StatefulSets, and new CronJobs"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: stbenjam
To complete the pull request process, please assign deads2k after the PR has been reviewed.
You can assign the PR to them by writing /assign @deads2k in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the vendor-update Touching vendor dir or related files label Nov 3, 2021
@openshift-ci openshift-ci bot requested review from marun and soltysh November 3, 2021 12:17
@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

@stbenjam: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/verify-commits f8a9910 link true /test verify-commits
ci/prow/e2e-gcp-upgrade f8a9910 link true /test e2e-gcp-upgrade
ci/prow/e2e-agnostic-cmd f8a9910 link false /test e2e-agnostic-cmd

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stbenjam
Copy link
Member Author

stbenjam commented Nov 3, 2021

/close

@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

@stbenjam: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot closed this Nov 3, 2021
@openshift-ci
Copy link

openshift-ci bot commented Nov 3, 2021

@stbenjam: This pull request references Bugzilla bug 2000216. The bug has been updated to no longer refer to the pull request using the external bug tracker.

In response to this:

[WIP] Revert "Bug 2000216: Image policy should mutate DeploymentConfigs, StatefulSets, and new CronJobs"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. vendor-update Touching vendor dir or related files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants