COO-362: [Chore] Add Cypress tests for Tracing UI plugin testing.

[IshwarKanse]

The PR adds Cypress tests for testing the Tracing UI plugin. Part of https://issues.redhat.com/browse/COO-362

Openshift Distributed Tracing UI Plugin Tests

These console tests install the Openshift Cluster Observability Operator with the Distributed Tracing UI Plugin, Tempo and OpenTelemetry Operators in the specified cluster and then run a series of tests against the UI.

Prerequisite

1. node.js >= 18
2. chainsaw >= v0.2.12

Install dependencies

All required dependencies are defined in package.json in order to run Cypress tests, run npm install so that dependencies will be installed in node_modules folder

$ npm install
$ ls -ltr
node_modules/     -> dependencies will be installed at runtime here

Directory structure

After dependencies are installed successfully and before we run actual tests, please confirm if we have correct structure as below.

% ls -ltr tests
drwxr-xr-x  views
-rw-r--r--  reporter-config.json
-rw-r--r--  package.json
drwxr-xr-x  node_modules
drwxr-xr-x  cypress
-rw-r--r--  cypress.config.ts
-rw-r--r--  README.md
drwxr-xr-x  tests
-rw-r--r--  tsconfig.json
drwxr-xr-x  fixtures

Export necessary variables

in order to run Cypress tests, we need to export some environment variables that Cypress can read then pass down to our tests, currently we have following environment variables defined and used.

Using a non-admin user.

export CYPRESS_BASE_URL=https://<console_route_spec_host>
export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider
export CYPRESS_LOGIN_USERS=username:password
export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig

Using kubeadmin user.

export CYPRESS_BASE_URL=https://<console_route_spec_host>
export CYPRESS_LOGIN_IDP=kube:admin
export CYPRESS_LOGIN_USERS=kubeadmin:password
export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig

Set the var to skip Cluster Observability and all the required operators installation.

export CYPRESS_SKIP_COO_INSTALL=true

Set the var to install Cluster Observability, OpenTelemetry and Tempo operators from redhat-operators catalog source.

export CYPRESS_COO_UI_INSTALL=true

Set the var to install Cluster Observability Operator using Konflux bundle. Tempo and OpenTelemetry operators will be installed from redhat-operators catalog source.

export CYPRESS_KONFLUX_COO_BUNDLE_IMAGE=<COO image>

Set the var to use custom Cluster Observability Operator bundle image. Tempo and OpenTelemetry operators will be installed from redhat-operators catalog source.

export CYPRESS_CUSTOM_COO_BUNDLE_IMAGE=<COO bundle image>

Set the following var to use custom Distributed Tracing UI plugin image. The image will be patched in Cluster Observability Operator CSV.

export CYPRESS_DT_CONSOLE_IMAGE=<console image>

Start Cypress

We can either open Cypress GUI(open) or run Cypress in headless mode(run) to run the tests.

npx cypress open
npx cypress run

There are currently two tests:

1. Create a Tracing UI plugin instance and check the Observe/Traces page is available.
2. Create a Tracing UI plugin instance, deploy TempoStack and TempoMonolithic, generate traces and verify the traces for both the Tempo instances from Tracing UI.

Test run.

`% npx cypress run

DevTools listening on ws://127.0.0.1:56223/devtools/browser/6d69f6c3-977d-4606-a3b8-04153ec67395

====================================================================================================

(Run Starting)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.17.0 │
│ Browser: Electron 118 (headless) │
│ Node Version: v20.11.0 (/usr/local/bin/node) │
│ Specs: 1 found (dt-plugin-tests.cy.ts) │
│ Searched: tests/**/*.cy.{js,jsx,ts,tsx} │
│ Experiments: experimentalMemoryManagement=true,experimentalModifyObstructiveThirdPartyCode… │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: dt-plugin-tests.cy.ts (1 of 1)

OpenShift Distributed Tracing UI Plugin tests

Logging in as kubeadmin
✓ (Test Distributed Tracing UI plugin page without any Tempo instances) (190337ms)
Command "oc delete UIPlugin distributed-tracing --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete project chainsaw-multitenancy chainsaw-monolithic-multitenancy --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-cluster-observability-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-opentelemetry-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-tempo-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc adm policy remove-cluster-role-from-user cluster-admin kubeadmin --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
✓ (Test Distributed Tracing UI plugin) (173086ms)

2 passing (7m)

[mochawesome] Report JSON saved to /Users/ikanse/distributed-tracing-console-plugin/tests/gui_test_screenshots/cypress_report_001.json

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 2 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 7 minutes, 28 seconds │
│ Spec Ran: dt-plugin-tests.cy.ts │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

====================================================================================================

(Run Finished)

   Spec                                              Tests  Passing  Failing  Pending  Skipped  

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ dt-plugin-tests.cy.ts 07:28 2 2 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 07:28 2 2 - - -
`

[IshwarKanse]

/retest

[IshwarKanse]

cc @yapei @yanpzhan

[openshift-ci]

@IshwarKanse: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[vprashar2929]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: IshwarKanse, vprashar2929

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [IshwarKanse]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@IshwarKanse: This pull request references COO-362 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.20.0" version, but no target version was set.

In response to this:

The PR adds Cypress tests for testing the Tracing UI plugin. Part of https://issues.redhat.com/browse/COO-362

Openshift Distributed Tracing UI Plugin Tests

These console tests install the Openshift Cluster Observability Operator with the Distributed Tracing UI Plugin, Tempo and OpenTelemetry Operators in the specified cluster and then run a series of tests against the UI.

Prerequisite

1. node.js >= 18
2. chainsaw >= v0.2.12

Install dependencies

All required dependencies are defined in package.json in order to run Cypress tests, run npm install so that dependencies will be installed in node_modules folder

$ npm install
$ ls -ltr
node_modules/     -> dependencies will be installed at runtime here

Directory structure

After dependencies are installed successfully and before we run actual tests, please confirm if we have correct structure as below.

% ls -ltr tests
drwxr-xr-x  views
-rw-r--r--  reporter-config.json
-rw-r--r--  package.json
drwxr-xr-x  node_modules
drwxr-xr-x  cypress
-rw-r--r--  cypress.config.ts
-rw-r--r--  README.md
drwxr-xr-x  tests
-rw-r--r--  tsconfig.json
drwxr-xr-x  fixtures

Export necessary variables

in order to run Cypress tests, we need to export some environment variables that Cypress can read then pass down to our tests, currently we have following environment variables defined and used.

Using a non-admin user.

export CYPRESS_BASE_URL=https://<console_route_spec_host>
export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider
export CYPRESS_LOGIN_USERS=username:password
export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig

Using kubeadmin user.

export CYPRESS_BASE_URL=https://<console_route_spec_host>
export CYPRESS_LOGIN_IDP=kube:admin
export CYPRESS_LOGIN_USERS=kubeadmin:password
export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig

Set the var to skip Cluster Observability and all the required operators installation.

export CYPRESS_SKIP_COO_INSTALL=true

Set the var to install Cluster Observability, OpenTelemetry and Tempo operators from redhat-operators catalog source.

export CYPRESS_COO_UI_INSTALL=true

Set the var to install Cluster Observability Operator using Konflux bundle. Tempo and OpenTelemetry operators will be installed from redhat-operators catalog source.

export CYPRESS_KONFLUX_COO_BUNDLE_IMAGE=<COO image>

Set the var to use custom Cluster Observability Operator bundle image. Tempo and OpenTelemetry operators will be installed from redhat-operators catalog source.

export CYPRESS_CUSTOM_COO_BUNDLE_IMAGE=<COO bundle image>

Set the following var to use custom Distributed Tracing UI plugin image. The image will be patched in Cluster Observability Operator CSV.

export CYPRESS_DT_CONSOLE_IMAGE=<console image>

Start Cypress

We can either open Cypress GUI(open) or run Cypress in headless mode(run) to run the tests.

npx cypress open
npx cypress run

There are currently two tests:

1. Create a Tracing UI plugin instance and check the Observe/Traces page is available.
2. Create a Tracing UI plugin instance, deploy TempoStack and TempoMonolithic, generate traces and verify the traces for both the Tempo instances from Tracing UI.

Test run.

`% npx cypress run

DevTools listening on ws://127.0.0.1:56223/devtools/browser/6d69f6c3-977d-4606-a3b8-04153ec67395

====================================================================================================

(Run Starting)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.17.0 │
│ Browser: Electron 118 (headless) │
│ Node Version: v20.11.0 (/usr/local/bin/node) │
│ Specs: 1 found (dt-plugin-tests.cy.ts) │
│ Searched: tests/**/*.cy.{js,jsx,ts,tsx} │
│ Experiments: experimentalMemoryManagement=true,experimentalModifyObstructiveThirdPartyCode… │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: dt-plugin-tests.cy.ts (1 of 1)

OpenShift Distributed Tracing UI Plugin tests

Logging in as kubeadmin
✓ (Test Distributed Tracing UI plugin page without any Tempo instances) (190337ms)
Command "oc delete UIPlugin distributed-tracing --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete project chainsaw-multitenancy chainsaw-monolithic-multitenancy --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-cluster-observability-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-opentelemetry-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-tempo-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc adm policy remove-cluster-role-from-user cluster-admin kubeadmin --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
✓ (Test Distributed Tracing UI plugin) (173086ms)

2 passing (7m)

[mochawesome] Report JSON saved to /Users/ikanse/distributed-tracing-console-plugin/tests/gui_test_screenshots/cypress_report_001.json

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 2 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 7 minutes, 28 seconds │
│ Spec Ran: dt-plugin-tests.cy.ts │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

====================================================================================================

(Run Finished)

  Spec                                              Tests  Passing  Failing  Pending  Skipped  

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ dt-plugin-tests.cy.ts 07:28 2 2 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 07:28 2 2 - - -
`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[jgbernalp]

/label ux-approved
/label qe-approved
no UX changes, QE improvements.

[openshift-ci-robot]

@IshwarKanse: This pull request references COO-362 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.20.0" version, but no target version was set.

In response to this:

The PR adds Cypress tests for testing the Tracing UI plugin. Part of https://issues.redhat.com/browse/COO-362

Openshift Distributed Tracing UI Plugin Tests

These console tests install the Openshift Cluster Observability Operator with the Distributed Tracing UI Plugin, Tempo and OpenTelemetry Operators in the specified cluster and then run a series of tests against the UI.

Prerequisite

1. node.js >= 18
2. chainsaw >= v0.2.12

Install dependencies

All required dependencies are defined in package.json in order to run Cypress tests, run npm install so that dependencies will be installed in node_modules folder

$ npm install
$ ls -ltr
node_modules/     -> dependencies will be installed at runtime here

Directory structure

After dependencies are installed successfully and before we run actual tests, please confirm if we have correct structure as below.

% ls -ltr tests
drwxr-xr-x  views
-rw-r--r--  reporter-config.json
-rw-r--r--  package.json
drwxr-xr-x  node_modules
drwxr-xr-x  cypress
-rw-r--r--  cypress.config.ts
-rw-r--r--  README.md
drwxr-xr-x  tests
-rw-r--r--  tsconfig.json
drwxr-xr-x  fixtures

Export necessary variables

in order to run Cypress tests, we need to export some environment variables that Cypress can read then pass down to our tests, currently we have following environment variables defined and used.

Using a non-admin user.

export CYPRESS_BASE_URL=https://<console_route_spec_host>
export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider
export CYPRESS_LOGIN_USERS=username:password
export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig

Using kubeadmin user.

export CYPRESS_BASE_URL=https://<console_route_spec_host>
export CYPRESS_LOGIN_IDP=kube:admin
export CYPRESS_LOGIN_USERS=kubeadmin:password
export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig

Set the var to skip Cluster Observability and all the required operators installation.

export CYPRESS_SKIP_COO_INSTALL=true

Set the var to install Cluster Observability, OpenTelemetry and Tempo operators from redhat-operators catalog source.

export CYPRESS_COO_UI_INSTALL=true

Set the var to install Cluster Observability Operator using Konflux bundle. Tempo and OpenTelemetry operators will be installed from redhat-operators catalog source.

export CYPRESS_KONFLUX_COO_BUNDLE_IMAGE=<COO image>

Set the var to use custom Cluster Observability Operator bundle image. Tempo and OpenTelemetry operators will be installed from redhat-operators catalog source.

export CYPRESS_CUSTOM_COO_BUNDLE_IMAGE=<COO bundle image>

Set the following var to use custom Distributed Tracing UI plugin image. The image will be patched in Cluster Observability Operator CSV.

export CYPRESS_DT_CONSOLE_IMAGE=<console image>

Start Cypress

We can either open Cypress GUI(open) or run Cypress in headless mode(run) to run the tests.

npx cypress open
npx cypress run

There are currently two tests:

1. Create a Tracing UI plugin instance and check the Observe/Traces page is available.
2. Create a Tracing UI plugin instance, deploy TempoStack and TempoMonolithic, generate traces and verify the traces for both the Tempo instances from Tracing UI.

Test run.

`% npx cypress run

DevTools listening on ws://127.0.0.1:56223/devtools/browser/6d69f6c3-977d-4606-a3b8-04153ec67395

====================================================================================================

(Run Starting)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.17.0 │
│ Browser: Electron 118 (headless) │
│ Node Version: v20.11.0 (/usr/local/bin/node) │
│ Specs: 1 found (dt-plugin-tests.cy.ts) │
│ Searched: tests/**/*.cy.{js,jsx,ts,tsx} │
│ Experiments: experimentalMemoryManagement=true,experimentalModifyObstructiveThirdPartyCode… │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: dt-plugin-tests.cy.ts (1 of 1)

OpenShift Distributed Tracing UI Plugin tests

Logging in as kubeadmin
✓ (Test Distributed Tracing UI plugin page without any Tempo instances) (190337ms)
Command "oc delete UIPlugin distributed-tracing --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete project chainsaw-multitenancy chainsaw-monolithic-multitenancy --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-cluster-observability-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-opentelemetry-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc delete namespace openshift-tempo-operator --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
Command "oc adm policy remove-cluster-role-from-user cluster-admin kubeadmin --kubeconfig /Users/ikanse/icloud-drive/ocp/cluster/auth/kubeconfig" executed successfully
✓ (Test Distributed Tracing UI plugin) (173086ms)

2 passing (7m)

[mochawesome] Report JSON saved to /Users/ikanse/distributed-tracing-console-plugin/tests/gui_test_screenshots/cypress_report_001.json

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 2 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 7 minutes, 28 seconds │
│ Spec Ran: dt-plugin-tests.cy.ts │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

====================================================================================================

(Run Finished)

  Spec                                              Tests  Passing  Failing  Pending  Skipped  

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ dt-plugin-tests.cy.ts 07:28 2 2 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 07:28 2 2 - - -
`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[IshwarKanse]

/cherry-pick release-0.4

[openshift-cherrypick-robot]

@IshwarKanse: new pull request created: #92

In response to this:

/cherry-pick release-0.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.