[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor… #242

openshift-cherrypick-robot · 2020-03-10T17:32:14Z

This is an automated cherry-pick of #240

/assign gabemontero

…terrors

openshift-ci-robot · 2020-03-10T17:32:17Z

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1811143, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.5.0" instead
expected the bug to be in one of the following states: NEW, ASSIGNED, ON_DEV, POST, POST, but it is ON_QA instead
expected Bugzilla bug 1811143 to depend on a bug targeting the "4.5.0" release and in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.4] Bug 1811143: purge removed imagestreams as a part of upgrade from progessing/impor…

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-10T17:32:17Z

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1811143, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.5.0" instead
expected the bug to be in one of the following states: NEW, ASSIGNED, ON_DEV, POST, POST, but it is ON_QA instead
expected Bugzilla bug 1811143 to depend on a bug targeting the "4.5.0" release and in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.4] Bug 1811143: purge removed imagestreams as a part of upgrade from progessing/impor…

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-10T17:34:21Z

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1811204, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.4.z" instead
expected dependent Bugzilla bug 1811143 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is ON_QA instead
expected dependent Bugzilla bug 1811206 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
expected dependent Bugzilla bug 1811206 to target the "4.5.0" release, but it targets "4.3.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor…

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-10T17:34:21Z

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1811204, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.4.z" instead
expected dependent Bugzilla bug 1811143 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is ON_QA instead
expected dependent Bugzilla bug 1811206 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
expected dependent Bugzilla bug 1811206 to target the "4.5.0" release, but it targets "4.3.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor…

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

gabemontero · 2020-03-12T13:53:43Z

CI lease issue on e2e-aws-operator

/test e2e-aws-operator

gabemontero · 2020-03-12T15:17:06Z

Problems pulling images in image-eco:

 Warning  Failed     8m25s (x4 over 9m58s)  kubelet, ip-10-0-135-231.ec2.internal  Failed to pull image "image-registry.openshift-image-registry.svc@sha256:ad5dc22e6115adc0d875f6d2eb44b2ba594d07330a600e67bf3de49e02cab5b0": rpc error: code = Unknown desc = Error reading manifest sha256:ad5dc22e6115adc0d875f6d2eb44b2ba594d07330a600e67bf3de49e02cab5b0 in docker.io/library/image-registry.openshift-image-registry.svc: errors:
denied: requested access to the resource is denied
unauthorized: authentication required

In talking with @dmage the docker.io/library/image-registry.openshift-image-registry.svc seems like a bad image ref.

Will debug further to see if I can backtrack where that comes from.

gabemontero · 2020-03-12T17:41:35Z

Ok, so that docker.io/library/... image ref came from pulling image-registry.openshift-image-registry.svc@sha256:ad5dc22e6115adc0d875f6d2eb44b2ba594d07330a600e67bf3de49e02cab5b0

That is the image associated with the mongodb DC in the test after a oc new-app mongodb-ephemeral

Based on https://github.com/openshift/cluster-samples-operator/blob/release-4.4/tmp/build/assets/operator/ocp-x86_64/mongodb/templates/mongodb-ephemeral.json#L202 that means the mongodb samples imagestream and version 3.6 based on https://github.com/openshift/cluster-samples-operator/blob/release-4.4/tmp/build/assets/operator/ocp-x86_64/mongodb/templates/mongodb-ephemeral.json#L274

According to the imagestream that means this TBR image: https://github.com/openshift/cluster-samples-operator/blob/release-4.4/tmp/build/assets/operator/ocp-x86_64/mongodb/imagestreams/mongodb-rhel7.json#L76

Going to the must-gather, the mongodb 3.6 IST status is this:

      tag: "3.6"
    - items:
      - created: "2020-03-10T18:33:14Z"
        dockerImageReference: registry.redhat.io/rhscl/mongodb-36-rhel7@sha256:ad5dc22e6115adc0d875f6d2eb44b2ba594d07330a600e67bf3de49e02cab5b0
        generation: 2
        image: sha256:ad5dc22e6115adc0d875f6d2eb44b2ba594d07330a600e67bf3de49e02cab5b0

the image sha sha256:ad5dc22e6115adc0d875f6d2eb44b2ba594d07330a600e67bf3de49e02cab5b0 matches what the kubelet tried to pull .... and the ref above certainly is going to registry.redhat.io not docker.io/library

I don't see any reference to that sha in the openshift-apiserver or image-registry pods.

@dmage how do you think we should proceed?

gabemontero · 2020-03-13T15:20:53Z

so @dmage was able to finish connecting the dots for us ... it is our old friend in e2e's ... the OCM already knows about the registry name, but the API server hasn't seen it yet issue we've seen with ICT builds and now in this case deployments.

I tried to fix this with openshift/openshift-controller-manager#56 but that was not sufficient

I'll be opening a new bug/PR shortly.

gabemontero · 2020-03-13T15:26:39Z

https://bugzilla.redhat.com/show_bug.cgi?id=1817099 for CI flake

gabemontero · 2020-03-13T15:53:00Z

Have openshift/openshift-controller-manager#73 up but let's see how persistent our old flake is

/retest

gabemontero · 2020-03-13T17:21:37Z

forbidden: unable to validate against any security context constraint is still around in 4.4 apparently in https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_cluster-samples-operator/242/pull-ci-openshift-cluster-samples-operator-release-4.4-e2e-aws-image-ecosystem/5

... it quit happening for us in 4.5/master

gabemontero · 2020-03-16T18:56:31Z

forbidden: unable to validate against any security context constraint is still around in 4.4 apparently in https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_cluster-samples-operator/242/pull-ci-openshift-cluster-samples-operator-release-4.4-e2e-aws-image-ecosystem/5

... it quit happening for us in 4.5/master

https://bugzilla.redhat.com/show_bug.cgi?id=1817099 was opened recently and seems to fit the bill in that
a) centers around vanilla pod resteart
b) happens "soon" after cluster provisioning, which these e2e failures could be classified as

gabemontero · 2020-03-16T19:42:01Z

/bugzilla refresh

openshift-ci-robot · 2020-03-16T19:42:05Z

@gabemontero: This pull request references Bugzilla bug 1811204, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.4.z" instead
expected dependent Bugzilla bug 1811206 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
expected dependent Bugzilla bug 1811206 to target the "4.5.0" release, but it targets "4.3.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-16T19:42:06Z

@gabemontero: This pull request references Bugzilla bug 1811204, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.4.z" instead
expected dependent Bugzilla bug 1811206 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
expected dependent Bugzilla bug 1811206 to target the "4.5.0" release, but it targets "4.3.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

gabemontero · 2020-03-16T19:55:40Z

/bugzilla refresh

openshift-ci-robot · 2020-03-16T19:55:43Z

@gabemontero: This pull request references Bugzilla bug 1811204, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.4.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-16T19:55:43Z

@gabemontero: This pull request references Bugzilla bug 1811204, which is invalid:

expected the bug to target the "4.4.0" release, but it targets "4.4.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

gabemontero · 2020-03-17T13:10:40Z

/retest

gabemontero · 2020-04-13T17:56:53Z

/bugzilla refresh

openshift-ci-robot · 2020-04-13T17:56:58Z

@gabemontero: This pull request references Bugzilla bug 1811204, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.4.0) matches configured target release for branch (4.4.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1811143 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1811143 targets the "4.5.0" release, which is one of the valid target releases: 4.5.0
bug has dependents

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-04-13T17:56:58Z

@gabemontero: This pull request references Bugzilla bug 1811204, which is valid.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.4.0) matches configured target release for branch (4.4.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1811143 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1811143 targets the "4.5.0" release, which is one of the valid target releases: 4.5.0
bug has dependents

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

adambkaplan

/lgtm

openshift-ci-robot · 2020-04-14T14:11:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [adambkaplan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-04-14T14:11:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [adambkaplan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2020-04-18T18:01:24Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-04-18T18:14:21Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-04-18T19:19:20Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-04-18T22:08:22Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-04-18T22:21:23Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-04-18T23:52:23Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2020-04-19T02:12:39Z

@openshift-cherrypick-robot: An error was encountered searching for bug 1811204 on the Bugzilla server at https://bugzilla.redhat.com:

Get https://bugzilla.redhat.com/rest/bug/1811204?api_key=CENSORED: dial tcp: i/o timeout
Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor…

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-04-19T02:12:41Z

@openshift-cherrypick-robot: An error was encountered searching for bug 1811204 on the Bugzilla server at https://bugzilla.redhat.com:

Get https://bugzilla.redhat.com/rest/bug/1811204?api_key=CENSORED: dial tcp: i/o timeout
Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor…

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

purge removed imagestreams as a part of upgrade from progessing/impor…

49f3aa1

…terrors

openshift-cherrypick-robot mentioned this pull request Mar 10, 2020

Bug 1811143: purge removed imagestreams as a part of upgrade from progessing/impor… #240

Merged

openshift-ci-robot assigned gabemontero Mar 10, 2020

openshift-ci-robot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Mar 10, 2020

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 10, 2020

openshift-ci-robot requested review from adambkaplan and coreydaley March 10, 2020 17:32

gabemontero changed the title ~~[release-4.4] Bug 1811143: purge removed imagestreams as a part of upgrade from progessing/impor…~~ [release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor… Mar 10, 2020

openshift-ci-robot requested a review from dmage March 10, 2020 17:35

gabemontero mentioned this pull request Mar 16, 2020

add scc forbidden error debug around scl image pod creation openshift/origin#24703

Closed

openshift-ci-robot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Apr 13, 2020

adambkaplan approved these changes Apr 14, 2020

View reviewed changes

openshift-ci-robot assigned adambkaplan Apr 14, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 14, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 14, 2020

bparees added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Apr 16, 2020

openshift-merge-robot merged commit 7dbd4ee into openshift:release-4.4 Apr 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor… #242

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor… #242

openshift-cherrypick-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

gabemontero commented Mar 12, 2020

gabemontero commented Mar 12, 2020

gabemontero commented Mar 12, 2020

gabemontero commented Mar 13, 2020

gabemontero commented Mar 13, 2020 •

edited

gabemontero commented Mar 13, 2020

gabemontero commented Mar 13, 2020 •

edited

gabemontero commented Mar 16, 2020 •

edited

gabemontero commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

gabemontero commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

gabemontero commented Mar 17, 2020

gabemontero commented Apr 13, 2020

openshift-ci-robot commented Apr 13, 2020

openshift-ci-robot commented Apr 13, 2020

adambkaplan left a comment

openshift-ci-robot commented Apr 14, 2020

openshift-ci-robot commented Apr 14, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-ci-robot commented Apr 19, 2020

openshift-ci-robot commented Apr 19, 2020

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor… #242

[release-4.4] Bug 1811204: purge removed imagestreams as a part of upgrade from progessing/impor… #242

Conversation

openshift-cherrypick-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

openshift-ci-robot commented Mar 10, 2020

gabemontero commented Mar 12, 2020

gabemontero commented Mar 12, 2020

gabemontero commented Mar 12, 2020

gabemontero commented Mar 13, 2020

gabemontero commented Mar 13, 2020 • edited

gabemontero commented Mar 13, 2020

gabemontero commented Mar 13, 2020 • edited

gabemontero commented Mar 16, 2020 • edited

gabemontero commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

gabemontero commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

openshift-ci-robot commented Mar 16, 2020

gabemontero commented Mar 17, 2020

gabemontero commented Apr 13, 2020

openshift-ci-robot commented Apr 13, 2020

openshift-ci-robot commented Apr 13, 2020

adambkaplan left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Apr 14, 2020

openshift-ci-robot commented Apr 14, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-bot commented Apr 18, 2020

openshift-ci-robot commented Apr 19, 2020

openshift-ci-robot commented Apr 19, 2020

gabemontero commented Mar 13, 2020 •

edited

gabemontero commented Mar 13, 2020 •

edited

gabemontero commented Mar 16, 2020 •

edited