Bug 2059515: e2e: Pin the Keycloak version used in tests #551

pierreprinetti · 2022-03-01T14:49:54Z

Before this patch, the e2e tests pulled the latest tag of
quay.io/keycloak/keycloak. The upgrade of the latest image to
Keycloack v17.0.0 made the e2e test suite fail.

More specifically, in the same conditions the 17.0.0 image does not
start automatically any more when the container is created.

This patch pins the Keycloak version to the "legacy" variant of v17.0.0,
which exposes the same behaviour as what the tests expect.

openshift-ci · 2022-03-01T14:50:05Z

@pierreprinetti: This pull request references Bugzilla bug 2059515, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

WIP: Bug 2059515: e2e: Pin the Keycloak version used in tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

pierreprinetti · 2022-03-01T16:50:14Z

In a separate patch, I will propose to instrument 4.11 to use the non-legacy variant of Keycloak v17, and keep some form of pinning.

/retitle Bug 2059515: e2e: Pin the Keycloak version used in tests
/bugzilla refresh
/cc s-urbaniak

openshift-ci · 2022-03-01T16:50:22Z

@pierreprinetti: This pull request references Bugzilla bug 2059515, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.11.0) matches configured target release for branch (4.11.0)
bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @xingxingxia

In response to this:

More specifically, the 17.0.0 image does not start automatically when the container is created.

Here I propose to pin the Keycloak version anyway and backport the change down to v4.7 to unblock any other patches. In a separate patch, we can instrument 4.11 to actually use Keycloak v17 (by passing the start or start-dev command).

One alternative can be to slap start-dev here and in the backports.

/retitle Bug 2059515: e2e: Pin the Keycloak version used in tests
/bugzilla refresh
/cc s-urbaniak

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2022-03-01T20:24:22Z

@pierreprinetti: This pull request references Bugzilla bug 2059515, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.11.0) matches configured target release for branch (4.11.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @xingxingxia

In response to this:

Bug 2059515: e2e: Pin the Keycloak version used in tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

pierreprinetti · 2022-03-01T22:59:28Z

/assign stlaz

xingxingxia · 2022-03-02T02:49:28Z

It was raised in keycloak/keycloak-quickstarts#300 but no update yet, @pierreprinetti FYI.

pierreprinetti · 2022-03-02T04:57:50Z

/retest

pierreprinetti · 2022-03-02T07:21:10Z

/retest
/cherry-pick release-4.10

openshift-cherrypick-robot · 2022-03-02T07:21:11Z

@pierreprinetti: once the present PR merges, I will cherry-pick it on top of release-4.10 in a new PR and assign it to you.

In response to this:

/retest
/cherry-pick release-4.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

s-urbaniak · 2022-03-02T09:43:35Z

/lgtm

openshift-ci · 2022-03-02T09:44:18Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pierreprinetti, s-urbaniak
To complete the pull request process, please ask for approval from stlaz after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pierreprinetti · 2022-03-02T09:51:42Z

/retest-required

pierreprinetti · 2022-03-02T12:33:45Z

/retest-required

stlaz · 2022-03-03T07:56:06Z

/hold
share:

issue in a Keycloak issue tracker that shows that the developers are aware of the problem. Latest openshift-examples/keycloak.yaml of 17.0.0 does not work keycloak/keycloak-quickstarts#300 is not enough because nobody has looked into it for 14 days. Contact the developers personally (we have the choice in RH, fortunately), to check they are aware of the issue and see when they are going to fix it if Latest openshift-examples/keycloak.yaml of 17.0.0 does not work keycloak/keycloak-quickstarts#300 is the only tracker for it
open a release-blocking BZ to move back to the latest image tag. We want to be testing with the latest Keycloak releases to make sure our functionality does not break with their new release

pierreprinetti · 2022-03-03T08:04:14Z

@stlaz
Thank you for the swift review.

As mentioned in the BZ, I have already contacted the Keycloak devs. They mentioned that v17 introduced an API change and there's no intention to come back to the old behaviour (see the internal conversation in the chat).

Since you don't want to pin the test dependency, I am going to change this PR to apply the alternative solution, which is: change the tests to use the new (v17, Quarkus-based) API.

My understanding is that you expect the API change to also be backported; that's what I am now going for.

openshift-ci · 2022-03-03T08:21:44Z

New changes are detected. LGTM label has been removed.

Before this patch, the e2e tests pulled the `latest` tag of `quay.io/keycloak/keycloak`. The upgrade of the `latest` image to Keycloack v17.0.0 made the e2e test suite fail. More specifically, in the same conditions the 17.0.0 image [does not start automatically][1] any more when the container is created. This patch pins the Keycloak version to the "legacy" variant of v17.0.0, which exposes the same behaviour as what the tests expect. [1]: keycloak/keycloak-containers@41baf7c

openshift-ci · 2022-03-03T10:11:32Z

@pierreprinetti: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-operator	`3f5f261`	link	true	`/test e2e-operator`
ci/prow/e2e-agnostic	`3f5f261`	link	true	`/test e2e-agnostic`
ci/prow/e2e-agnostic-ipv6	`3f5f261`	link	false	`/test e2e-agnostic-ipv6`
ci/prow/e2e-agnostic-upgrade	`3f5f261`	link	true	`/test e2e-agnostic-upgrade`
ci/prow/e2e-aws-single-node	`3f5f261`	link	false	`/test e2e-aws-single-node`
ci/prow/e2e-console-login	`3f5f261`	link	true	`/test e2e-console-login`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

pierreprinetti · 2022-03-03T15:07:08Z

/close
Superseded by #554

openshift-ci · 2022-03-03T15:13:30Z

@pierreprinetti: Closed this PR.

In response to this:

/close
Superseded by #554

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2022-03-03T15:13:35Z

@pierreprinetti: This pull request references Bugzilla bug 2059515. The bug has been updated to no longer refer to the pull request using the external bug tracker.

In response to this:

Bug 2059515: e2e: Pin the Keycloak version used in tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci bot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Mar 1, 2022

openshift-ci bot requested review from stlaz and sttts March 1, 2022 14:53

openshift-ci bot requested a review from s-urbaniak March 1, 2022 16:50

openshift-ci bot changed the title ~~WIP: Bug 2059515: e2e: Pin the Keycloak version used in tests~~ Bug 2059515: e2e: Pin the Keycloak version used in tests Mar 1, 2022

openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Mar 1, 2022

openshift-ci bot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Mar 1, 2022

openshift-ci bot requested a review from xingxingxia March 1, 2022 16:50

pierreprinetti force-pushed the bug2059515 branch 3 times, most recently from 26ab13b to 6439073 Compare March 1, 2022 20:23

pierreprinetti mentioned this pull request Mar 1, 2022

Bug 2052467: Custom route HTTPS certificate SAN validation #545

Merged

openshift-ci bot assigned stlaz Mar 1, 2022

openshift-ci bot assigned s-urbaniak Mar 2, 2022

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 2, 2022

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 3, 2022

pierreprinetti force-pushed the bug2059515 branch from 6439073 to 201d185 Compare March 3, 2022 08:21

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Mar 3, 2022

pierreprinetti force-pushed the bug2059515 branch from 201d185 to e9d07bf Compare March 3, 2022 09:03

pierreprinetti force-pushed the bug2059515 branch from e9d07bf to 3f5f261 Compare March 3, 2022 09:04

openshift-ci bot closed this Mar 3, 2022

Bug 2059515: e2e: Pin the Keycloak version used in tests #551

Bug 2059515: e2e: Pin the Keycloak version used in tests #551

Uh oh!

Conversation

pierreprinetti commented Mar 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Mar 1, 2022

Uh oh!

pierreprinetti commented Mar 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Mar 1, 2022

Uh oh!

openshift-ci bot commented Mar 1, 2022

Uh oh!

pierreprinetti commented Mar 1, 2022

Uh oh!

xingxingxia commented Mar 2, 2022

Uh oh!

pierreprinetti commented Mar 2, 2022

Uh oh!

pierreprinetti commented Mar 2, 2022

Uh oh!

openshift-cherrypick-robot commented Mar 2, 2022

Uh oh!

s-urbaniak commented Mar 2, 2022

Uh oh!

openshift-ci bot commented Mar 2, 2022

Uh oh!

pierreprinetti commented Mar 2, 2022

Uh oh!

pierreprinetti commented Mar 2, 2022

Uh oh!

stlaz commented Mar 3, 2022

Uh oh!

pierreprinetti commented Mar 3, 2022

Uh oh!

openshift-ci bot commented Mar 3, 2022

Uh oh!

openshift-ci bot commented Mar 3, 2022

Uh oh!

pierreprinetti commented Mar 3, 2022

Uh oh!

openshift-ci bot commented Mar 3, 2022

Uh oh!

openshift-ci bot commented Mar 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pierreprinetti commented Mar 1, 2022 •

edited

Loading

pierreprinetti commented Mar 1, 2022 •

edited

Loading