Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1788612: temp disable of openjdk-8-11 while it seems busted on registry.redhat.io #212

Merged

Conversation

gabemontero
Copy link
Contributor

I've seen the import fail in multiple installs since last Friday, as have others in 4-dev-triage, etc. and docker pulls from my laptop are proving very very flaky

@openshift/openshift-team-developer-experience @bparees fyi

/assign @adambkaplan

after this merges I'll craft a re-enablement PR and between that and local pulls see if/when it stabilizes

it is still listed as a valid option in the container catalog

@openshift-ci-robot openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 7, 2020
@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 7, 2020
@bparees
Copy link
Contributor

bparees commented Jan 7, 2020

after this merges I'll craft a re-enablement PR and between that and local pulls see if/when it stabilizes

please make sure we have a tracking bug to get it added back in as well. we don't want to ship this way if we don't have to.

(also is this impacting 4.3 testing?)

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 7, 2020
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bparees, gabemontero

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [bparees,gabemontero]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

19 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@adambkaplan
Copy link
Contributor

/hold

This seems to have been bit by the PHP EOL change, too:

event for cakephp-mysql-example: {buildconfig-controller } BuildConfigInstantiateFailed: error instantiating Build from BuildConfig e2e-test-cakephp-mysql-repo-test-4t25j/cakephp-mysql-example (0): Error resolving ImageStreamTag php:7.3 in namespace openshift: unable to find latest tagged image

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 7, 2020
@gabemontero
Copy link
Contributor Author

/hold

This seems to have been bit by the PHP EOL change, too:

event for cakephp-mysql-example: {buildconfig-controller } BuildConfigInstantiateFailed: error instantiating Build from BuildConfig e2e-test-cakephp-mysql-repo-test-4t25j/cakephp-mysql-example (0): Error resolving ImageStreamTag php:7.3 in namespace openshift: unable to find latest tagged image

The PHP EOL fix is in #211 which has green tests and is awaiting review/LGTM

@gabemontero
Copy link
Contributor Author

gabemontero commented Jan 7, 2020

after this merges I'll craft a re-enablement PR and between that and local pulls see if/when it stabilizes

please make sure we have a tracking bug to get it added back in as well. we don't want to ship this way if we don't have to.

(also is this impacting 4.3 testing?)

This is only 4.4 specific. The openjdk imagestream defs changed within the last month @bparees

In 4.3 we have registry.redhat.io/redhat-openjdk-18/openjdk18-openshift:1.1 and that seems to still import reliably. I took a quick scan of some of the 4.3 PR jobs must-gather data and confirmed that exists, and not registry.redhat.io/openjdk/openjdk-8-rhel8:1.1

You can also see the difference when looking at the 4.3 imagestream files at
https://github.com/openshift/cluster-samples-operator/tree/release-4.3/tmp/build/assets/operator/ocp-x86_64/java/imagestreams
vs. the 4.4/master files at
https://github.com/openshift/cluster-samples-operator/tree/master/tmp/build/assets/operator/ocp-x86_64/java/imagestreams

And to reiterate, the registry.redhat.io/openjdk/openjdk-8-rhel8:1.1 does exist. After several attempts, I was finally able to pull it onto my laptop while here in the office:

registry.redhat.io/openjdk/openjdk-8-rhel8 1.1 67569356870e 3 weeks ago 474MB

and the 1.0 tag seems to pull reliably.

This really feels like another of those PITA registry caching issues we've seen in the past. Or a TBR reliability bug of some sort. Though, I wonder if it is a caching issue, how much external location like AWS/GCP is affected by this, as a scan of 4.4 jobs on prow seems to show it frequently.

This query produced some hits

https://search.svc.ci.openshift.org/?search=Internal+error+occurred%3A+registry.redhat.io%2Fopenjdk%2Fopenjdk-1.8-rhel8%3A1.1&maxAge=168h&context=2&type=all

Though I do not think search ci can find data in must gather like

https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_release/6574/rehearse-6574-pull-ci-openshift-service-ca-operator-release-4.4-e2e-aws-upgrade/1/artifacts/e2e-aws-upgrade/must-gather/registry-svc-ci-openshift-org-ci-op-81mm211x-stable-sha256-dd03a9e0ac7b6e710037d8f0d0a5001b47c27bf859e49b0256e8ac54f5bd8198/namespaces/openshift/image.openshift.io/imagestreams.yaml

/lgtm

@bparees
Copy link
Contributor

bparees commented Jan 7, 2020

cool, thanks for the explanation @gabemontero!

@gabemontero gabemontero changed the title temp disable of openjdk-8-11 while it seems busted on registry.redhat.io Bug 1788612: temp disable of openjdk-8-11 while it seems busted on registry.redhat.io Jan 7, 2020
@openshift-ci-robot
Copy link
Contributor

@gabemontero: This pull request references Bugzilla bug 1788612, which is invalid:

  • expected the bug to target the "4.4.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1788612: temp disable of openjdk-8-11 while it seems busted on registry.redhat.io

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

1 similar comment
@openshift-ci-robot
Copy link
Contributor

@gabemontero: This pull request references Bugzilla bug 1788612, which is invalid:

  • expected the bug to target the "4.4.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1788612: temp disable of openjdk-8-11 while it seems busted on registry.redhat.io

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jan 7, 2020
@gabemontero
Copy link
Contributor Author

/bugzilla refresh

@openshift-ci-robot openshift-ci-robot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Jan 7, 2020
@openshift-ci-robot
Copy link
Contributor

@gabemontero: This pull request references Bugzilla bug 1788612, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Contributor

@gabemontero: This pull request references Bugzilla bug 1788612, which is valid. The bug has been moved to the POST state.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gabemontero
Copy link
Contributor Author

transferred PHP EOL to #213

@gabemontero
Copy link
Contributor Author

/hold cancel

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 8, 2020
@gabemontero
Copy link
Contributor Author

/retest

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

7 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit d48fc35 into openshift:master Jan 8, 2020
@openshift-ci-robot
Copy link
Contributor

@gabemontero: All pull requests linked via external trackers have merged. Bugzilla bug 1788612 has been moved to the MODIFIED state.

In response to this:

Bug 1788612: temp disable of openjdk-8-11 while it seems busted on registry.redhat.io

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

1 similar comment
@openshift-ci-robot
Copy link
Contributor

@gabemontero: All pull requests linked via external trackers have merged. Bugzilla bug 1788612 has been moved to the MODIFIED state.

In response to this:

Bug 1788612: temp disable of openjdk-8-11 while it seems busted on registry.redhat.io

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gabemontero gabemontero deleted the disable-openjdk-11 branch January 8, 2020 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants