CONSOLE-3740: Upgrade Cypress #13070

rhamilto · 2023-08-03T15:38:42Z

Based on my findings, there are a number of recommendations/observations:

As part of the upgrade from 8.5.0 >= 10.x, the existing cypress.json configs and test names need to be updated. Running cypress open with a >= 10.x version of Cypress installed provides a wizard that covers this process. I have done already done so for packages/integration-tests-cypress and packages/operator-lifecycle-manager/integration-tests-cypress in this PR. But the following configs and tests need updating:

packages/topology/integration-tests/cypress.json
packages/dev-console/integration-tests/cypress.json
packages/helm-plugin/integration-tests/cypress.json
packages/gitops-plugin/integration-tests /cypress.json
packages/knative-plugin/integration-tests/cypress.json
packages/pipelines-plugin/integration-tests /cypress.json
packages/shipwright-plugin/integration-tests/cypress.json
packages/webterminal-plugin/integration-tests/cypress.json
~~packages/ceph-storage-plugin/integration-tests-cypress/cypress-ceph.json~~ (will be removed per @SanjalKatiyar)

The tests run significantly faster in Cypress >= 10.x. This exposes an issue with tests that use cy.visit where the console redirects to / if the console isn't fully loaded before Cypress starts testing. A workaround is to visit / first and then check the perspective is set to Administrator. This appears to give the console time to fully load. I've made these adjustments for packages/integration-tests-cypress and packages/operator-lifecycle-manager/integration-tests-cypress via global command (cy.initAdmin) I added.

I've also added cy.createProjectWithCLI and cy.deleteProjectWithCLI. It occurs to me we should use the CLI to create and delete projects where testing these functions isn't important to the test in order to reduce the run time of the tests, which is a task from https://issues.redhat.com/browse/CONSOLE-3718. cc: @TheRealJon

Notes on updated tests

Requires demo plugin be running locally:

app/demo-dynamic-plugin.cy.ts

Require monitoring plugin be running locally:

crud/other-routes.cy.ts
i18n/pseudolocalization.cy.ts
monitoring/monitoring.cy.ts

frontend/.eslintignore

rhamilto · 2023-08-03T18:40:33Z

frontend/tsconfig.json

@@ -12,8 +12,15 @@
    "sourceMap": true,
    "noUnusedLocals": true,
    "typeRoots": ["node_modules/@types", "@types"],
-    "types": ["node", "jest", "cypress", "cypress-axe", "cypress-file-upload", "console"]
+    "types": ["node", "jest", "cypress-axe", "console"]


cypress and cypress-file-upload do not have @types packages.

frontend/tsconfig.json

rhamilto · 2023-08-24T11:27:20Z

/retest

rhamilto · 2023-08-25T12:26:37Z

/retest

openshift-merge-robot · 2023-10-03T02:25:14Z

@rhamilto: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-console	`d5cc206`	link	true	`/test e2e-gcp-console`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

TheRealJon

Just a few observations. I don't fully understand why the ts config for the integration test package is weird. I remember trying to tinker with it at one point because VSCode kept telling me that 'cy' wasn't defined in the global scope, but I didn't have much luck with it. Let me know if you want to look at it together.

README.md

frontend/packages/integration-tests-cypress/cypress.config.js

frontend/packages/integration-tests-cypress/support/login.ts

frontend/packages/integration-tests-cypress/support/admin.ts

frontend/packages/integration-tests-cypress/support/selectors.ts

frontend/packages/integration-tests-cypress/tsconfig.json

frontend/packages/operator-lifecycle-manager/integration-tests-cypress/tests/operator-hub.cy.ts

frontend/packages/operator-lifecycle-manager/integration-tests-cypress/views/operator.view.ts

frontend/tsconfig.json

RickJWagner

Looks good. Adding px-approved label.

opayne1 · 2023-10-09T19:16:12Z

/label docs-approved

openshift-ci-robot · 2023-10-09T19:16:20Z

@rhamilto: This pull request references CONSOLE-3740 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but it targets "openshift-4.15" instead.

In response to this:

Based on my findings, there are a number of recommendations/observations:

As part of the upgrade from 8.5.0 >= 10.x, the existing cypress.json configs and test names need to be updated. Running cypress open with a >= 10.x version of Cypress installed provides a wizard that covers this process. I have done already done so for packages/integration-tests-cypress and packages/operator-lifecycle-manager/integration-tests-cypress in this PR. But the following configs and tests need updating:

packages/topology/integration-tests/cypress.json

packages/dev-console/integration-tests/cypress.json

packages/helm-plugin/integration-tests/cypress.json

packages/gitops-plugin/integration-tests /cypress.json

packages/knative-plugin/integration-tests/cypress.json

packages/pipelines-plugin/integration-tests /cypress.json

packages/shipwright-plugin/integration-tests/cypress.json

packages/webterminal-plugin/integration-tests/cypress.json

~~packages/ceph-storage-plugin/integration-tests-cypress/cypress-ceph.json~~ (will be removed per @SanjalKatiyar)

The tests run significantly faster in Cypress >= 10.x. This exposes an issue with tests that use cy.visit where the console redirects to / if the console isn't fully loaded before Cypress starts testing. A workaround is to visit / first and then check the perspective is set to Administrator. This appears to give the console time to fully load. I've made these adjustments for packages/integration-tests-cypress and packages/operator-lifecycle-manager/integration-tests-cypress via global command (cy.initAdmin) I added.

I've also added cy.createProjectWithCLI and cy.deleteProjectWithCLI. It occurs to me we should use the CLI to create and delete projects where testing these functions isn't important to the test in order to reduce the run time of the tests, which is a task from https://issues.redhat.com/browse/CONSOLE-3718. cc: @TheRealJon

Notes on updated tests

Requires demo plugin be running locally:

app/demo-dynamic-plugin.cy.ts

Require monitoring plugin be running locally:

crud/other-routes.cy.ts

i18n/pseudolocalization.cy.ts

monitoring/monitoring.cy.ts

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

…in-Add-dashboard-card-with-chart-using-QueryBrowser-component CONSOLE-3675: Dynamic Demo Plugin: Add dashboard card with chart using QueryBrowser component

rhamilto · 2023-10-10T16:08:31Z

/retest

TheRealJon

Just a few comments. Let me know your thoughts.

frontend/.eslintignore

frontend/packages/integration-tests-cypress/tests/app/auth-multiuser-login.cy.ts

TheRealJon · 2023-10-12T17:09:55Z

frontend/packages/integration-tests-cypress/tests/app/masthead.cy.ts

+        cy.byTestID('user-dropdown').click();
+        cy.byTestID('log-out').should('be.visible');
+        cy.byTestID('log-out').click({ force: true });
+        cy.visit('/');


We need some kind of assertion here to make sure we actually logged out.

Probably, but this isn't functionally different than what it was doing before, so no change in behavior. Follow on?

Yeah, I think that's fine. I just worry that with sessions, we won't know if the logout failed.

So after playing with this, I don't think there is a way to actually test logout fully because of the way cy.session works since it's interecepting the credentials. Will leave it for now.

frontend/packages/integration-tests-cypress/tests/app/poll-console-updates.cy.ts

frontend/packages/integration-tests-cypress/support/login.ts

...packages/operator-lifecycle-manager/integration-tests-cypress/tests/operator-uninstall.cy.ts

TheRealJon

/lgtm

We can fix the logout test as a follow-on.

TheRealJon · 2023-10-12T21:07:39Z

frontend/packages/integration-tests-cypress/tests/app/masthead.cy.ts

+        cy.byTestID('user-dropdown').click();
+        cy.byTestID('log-out').should('be.visible');
+        cy.byTestID('log-out').click({ force: true });
+        cy.visit('/');


Yeah, I think that's fine. I just worry that with sessions, we won't know if the logout failed.

openshift-ci · 2023-10-12T21:10:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rhamilto, TheRealJon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [TheRealJon,rhamilto]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

rhamilto · 2023-10-12T21:52:08Z

Adding qe-approval label since this isn't customer facing.

/label qe-approval

openshift-ci · 2023-10-12T21:52:10Z

@rhamilto: The label(s) /label qe-approval cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, downstream-change-needed, rebase/manual, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, jira/valid-bug, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

Adding qe-approval label since this isn't customer facing.

/label qe-approval

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

rhamilto · 2023-10-12T21:53:06Z

/label qe-approved

typing is hard

openshift-ci-robot · 2023-10-12T21:53:13Z

@rhamilto: This pull request references CONSOLE-3740 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but it targets "openshift-4.15" instead.

In response to this:

Based on my findings, there are a number of recommendations/observations:

As part of the upgrade from 8.5.0 >= 10.x, the existing cypress.json configs and test names need to be updated. Running cypress open with a >= 10.x version of Cypress installed provides a wizard that covers this process. I have done already done so for packages/integration-tests-cypress and packages/operator-lifecycle-manager/integration-tests-cypress in this PR. But the following configs and tests need updating:

packages/topology/integration-tests/cypress.json

packages/dev-console/integration-tests/cypress.json

packages/helm-plugin/integration-tests/cypress.json

packages/gitops-plugin/integration-tests /cypress.json

packages/knative-plugin/integration-tests/cypress.json

packages/pipelines-plugin/integration-tests /cypress.json

packages/shipwright-plugin/integration-tests/cypress.json

packages/webterminal-plugin/integration-tests/cypress.json

~~packages/ceph-storage-plugin/integration-tests-cypress/cypress-ceph.json~~ (will be removed per @SanjalKatiyar)

The tests run significantly faster in Cypress >= 10.x. This exposes an issue with tests that use cy.visit where the console redirects to / if the console isn't fully loaded before Cypress starts testing. A workaround is to visit / first and then check the perspective is set to Administrator. This appears to give the console time to fully load. I've made these adjustments for packages/integration-tests-cypress and packages/operator-lifecycle-manager/integration-tests-cypress via global command (cy.initAdmin) I added.

I've also added cy.createProjectWithCLI and cy.deleteProjectWithCLI. It occurs to me we should use the CLI to create and delete projects where testing these functions isn't important to the test in order to reduce the run time of the tests, which is a task from https://issues.redhat.com/browse/CONSOLE-3718. cc: @TheRealJon

Notes on updated tests

Requires demo plugin be running locally:

app/demo-dynamic-plugin.cy.ts

Require monitoring plugin be running locally:

crud/other-routes.cy.ts

i18n/pseudolocalization.cy.ts

monitoring/monitoring.cy.ts

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-10-13T02:55:38Z

/retest-required

Remaining retests: 0 against base HEAD 7d79a03 and 2 for PR HEAD b269f98 in total

openshift-ci · 2023-10-13T05:58:40Z

@rhamilto: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 3, 2023

openshift-ci bot requested review from invincibleJai and jcaianirh August 3, 2023 15:39

openshift-ci bot added component/dev-console Related to dev-console component/olm Related to OLM approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/cypress Related to Cypress e2e integration testing labels Aug 3, 2023

rhamilto force-pushed the CONSOLE-3716 branch from 3d2cfd7 to 9a0bb40 Compare August 16, 2023 18:32

rhamilto commented Aug 16, 2023

View reviewed changes

rhamilto force-pushed the CONSOLE-3716 branch 3 times, most recently from 3d635b0 to b3042a1 Compare August 23, 2023 19:05

rhamilto changed the title ~~[WIP] Upgrade Cypress to v10.11.0~~ [WIP] Upgrade Cypress to v11.2.0 Aug 23, 2023

rhamilto mentioned this pull request Aug 24, 2023

OCPBUGS-10562: re-enable operator-uninstall.spec.ts but disable the failing test #12738

Closed

rhamilto force-pushed the CONSOLE-3716 branch from b3042a1 to 2d5d9b3 Compare August 24, 2023 17:30

rhamilto force-pushed the CONSOLE-3716 branch 2 times, most recently from b1ec710 to 439692b Compare September 27, 2023 18:27

openshift-ci bot added the component/topology Related to topology label Oct 2, 2023

rhamilto force-pushed the CONSOLE-3716 branch from d5cc206 to 6b11bbb Compare October 3, 2023 15:04

rhamilto changed the title ~~[WIP] Upgrade Cypress to v11.2.0~~ [WIP] Upgrade Cypress Oct 3, 2023

rhamilto force-pushed the CONSOLE-3716 branch 2 times, most recently from fcf1ff3 to c61c7ee Compare October 3, 2023 18:01

TheRealJon reviewed Oct 3, 2023

View reviewed changes

rhamilto force-pushed the CONSOLE-3716 branch from c61c7ee to 925abcc Compare October 4, 2023 18:13

rhamilto mentioned this pull request Oct 4, 2023

OCPBUGS-10562: fix operator uninstall test #13214

Merged

rhamilto force-pushed the CONSOLE-3716 branch 2 times, most recently from d08018d to 8b38367 Compare October 4, 2023 19:20

RickJWagner removed their assignment Oct 9, 2023

RickJWagner reviewed Oct 9, 2023

View reviewed changes

openshift-ci bot added the docs-approved Signifies that Docs has signed off on this PR label Oct 9, 2023

rhamilto referenced this pull request Oct 10, 2023

Merge pull request #13105 from Mylanos/CONSOLE-3675-Dynamic-Demo-Plug…

5002947

…in-Add-dashboard-card-with-chart-using-QueryBrowser-component CONSOLE-3675: Dynamic Demo Plugin: Add dashboard card with chart using QueryBrowser component

rhamilto added 3 commits October 10, 2023 07:28

Upgrade Cypress to v11.2.0

ce2ba13

Upgrade Cypress to v12.17.4

0c4314e

Disable other packages

b269f98

rhamilto force-pushed the CONSOLE-3716 branch from 1d48dc7 to b269f98 Compare October 10, 2023 13:02

vikram-raj mentioned this pull request Oct 11, 2023

ODC-7400: Fix e2e for dev-console, knative, topology, helm and pipeline packages #13231

Merged

rhamilto changed the title ~~[WIP] CONSOLE-3740: Upgrade Cypress~~ CONSOLE-3740: Upgrade Cypress Oct 12, 2023

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 12, 2023

TheRealJon reviewed Oct 12, 2023

View reviewed changes

TheRealJon approved these changes Oct 12, 2023

View reviewed changes

openshift-ci bot assigned TheRealJon Oct 12, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 12, 2023

ciiay mentioned this pull request Oct 12, 2023

GITOPS-3433: get ready for cypress upgrade #13239

Merged

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Oct 12, 2023

openshift-ci bot merged commit 1083040 into openshift:master Oct 13, 2023
6 checks passed

rhamilto deleted the CONSOLE-3716 branch October 13, 2023 12:01

rhamilto mentioned this pull request Oct 13, 2023

CONSOLE-3740: follow-on fixes and cleanup for Cypress upgrade #13242

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CONSOLE-3740: Upgrade Cypress #13070

CONSOLE-3740: Upgrade Cypress #13070

rhamilto commented Aug 3, 2023 •

edited

rhamilto Aug 3, 2023

rhamilto commented Aug 24, 2023

rhamilto commented Aug 25, 2023

openshift-merge-robot commented Oct 3, 2023

TheRealJon left a comment

RickJWagner left a comment

opayne1 commented Oct 9, 2023

openshift-ci-robot commented Oct 9, 2023 •

edited by openshift-ci bot

Notes on updated tests

rhamilto commented Oct 10, 2023

TheRealJon left a comment

TheRealJon Oct 12, 2023

rhamilto Oct 12, 2023

TheRealJon Oct 12, 2023

rhamilto Oct 13, 2023

TheRealJon left a comment

TheRealJon Oct 12, 2023

openshift-ci bot commented Oct 12, 2023

rhamilto commented Oct 12, 2023

openshift-ci bot commented Oct 12, 2023

rhamilto commented Oct 12, 2023

openshift-ci-robot commented Oct 12, 2023 •

edited by openshift-ci bot

Notes on updated tests

openshift-ci-robot commented Oct 13, 2023

openshift-ci bot commented Oct 13, 2023

CONSOLE-3740: Upgrade Cypress #13070

CONSOLE-3740: Upgrade Cypress #13070

Conversation

rhamilto commented Aug 3, 2023 • edited

Notes on updated tests

rhamilto Aug 3, 2023

Choose a reason for hiding this comment

rhamilto commented Aug 24, 2023

rhamilto commented Aug 25, 2023

openshift-merge-robot commented Oct 3, 2023

TheRealJon left a comment

Choose a reason for hiding this comment

RickJWagner left a comment

Choose a reason for hiding this comment

opayne1 commented Oct 9, 2023

openshift-ci-robot commented Oct 9, 2023 • edited by openshift-ci bot

Notes on updated tests

rhamilto commented Oct 10, 2023

TheRealJon left a comment

Choose a reason for hiding this comment

TheRealJon Oct 12, 2023

Choose a reason for hiding this comment

rhamilto Oct 12, 2023

Choose a reason for hiding this comment

TheRealJon Oct 12, 2023

Choose a reason for hiding this comment

rhamilto Oct 13, 2023

Choose a reason for hiding this comment

TheRealJon left a comment

Choose a reason for hiding this comment

TheRealJon Oct 12, 2023

Choose a reason for hiding this comment

openshift-ci bot commented Oct 12, 2023

rhamilto commented Oct 12, 2023

openshift-ci bot commented Oct 12, 2023

rhamilto commented Oct 12, 2023

openshift-ci-robot commented Oct 12, 2023 • edited by openshift-ci bot

Notes on updated tests

openshift-ci-robot commented Oct 13, 2023

openshift-ci bot commented Oct 13, 2023

rhamilto commented Aug 3, 2023 •

edited

openshift-ci-robot commented Oct 9, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Oct 12, 2023 •

edited by openshift-ci bot