Change shell and yarn scripts from test-gui to test-protractor. U…

…pdated README.md

README.md

@@ -188,18 +188,37 @@ Run frontend tests:
 
 #### Cypress
 
-Cypress integration tests are run via [Cypress.io](https://www.cypress.io/).
+Cypress integration tests are implemented in [Cypress.io](https://www.cypress.io/).
 
 Launch Cypress test runner:
 ```
 cd frontend
 oc login ...
-yarn run test-cypress
+yarn run test-cypress-console
 ```
 
-This will launch the Cypress test runner where you can run one or all cypress tests.
+This will launch the Cypress Test Runner UI in the `console` package, where you can run one or all cypress tests.
 By default, it will look for Chrome in the system and use it, but if you want to use Firefox instead, set `BRIDGE_E2E_BROWSER_NAME` environment variable in your shell with the value `firefox`.
 
+##### Execute Cypress in different packages
+
+An alternate way to execute cypress tests is via [test-cypress.sh](test-cypress.sh) which takes a `-p <package>` parameter to allow execution in different packages.  It also can run Cypress tests in the Test Runner UI or in `-- headless` mode:
+```
+console>./test-cypress.sh
+Runs Cypress tests in Test Runner or headless mode
+Usage: test-cypress [-p] <package> [-s] <filemask> [-h true]
+  '-p <package>' may be 'console, 'olm' or 'devconsole'
+  '-s <specmask>' is a file mask for spec test files, such as 'tests/monitoring/*'. Used only in headless mode when '-p' is specified.
+  '-h true' runs Cypress in headless mode. When omitted, launches Cypress Test Runner
+Examples:
+  test-cypress.sh                                       // displays this help text
+  test-cypress.sh -p console                            // opens Cypress Test Runner for console tests
+  test-cypress.sh -p olm                                // opens Cypress Test Runner for OLM tests
+  test-cypress.sh -h true                               // runs all packages in headless mode
+  test-cypress.sh -p olm -h true                        // runs OLM tests in headless mode
+  test-cypress.sh -p console -s 'tests/crud/*' -h true  // runs console CRUD tests in headless mode
+```
+
 [**_More information on Console's Cypress usage_**](frontend/packages/integration-tests-cypress/README.md)
 
 #### Protractor
@@ -215,12 +234,12 @@ cd frontend && yarn run webdriver-update
 
 Run integration tests:
 ```
-yarn run test-gui
+yarn run test-protractor
 ```
 
 Run integration tests on an OpenShift cluster:
 ```
-yarn run test-gui-openshift
+yarn run test-protractor-openshift
 ```
 This will include the normal k8s CRUD tests and CRUD tests for OpenShift
 resources.
@@ -239,55 +258,31 @@ For macOS, you can use:
 ```
 yarn run webdriver-update-macos
 ```
-#### How the Integration Tests Run in CI
-
-The end-to-end tests run against pull requests using [ci-operator](https://github.com/openshift/ci-operator/).
-The tests are defined in [this manifest](https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/console/openshift-console-master-presubmits.yaml)
-in the [openshift/release](https://github.com/openshift/release) repo and were generated with [ci-operator-prowgen](https://github.com/openshift/ci-operator-prowgen).
-
-CI runs the [test-prow-e2e.sh](test-prow-e2e.sh) script, which uses the `e2e` suite defined in [protractor.conf.ts](frontend/integration-tests/protractor.conf.ts).
-
-You can simulate an e2e run against an existing 4.0 cluster with the following commands (replace `/path/to/install-dir` with your OpenShift 4.0 install directory):
-
-```
-$ oc apply -f ./frontend/integration-tests/data/htpasswd-secret.yaml
-$ oc patch oauths cluster --patch "$(cat ./frontend/integration-tests/data/patch-htpasswd.yaml)" --type=merge
-$ export BRIDGE_BASE_ADDRESS="$(oc get consoles.config.openshift.io cluster -o jsonpath='{.status.consoleURL}')"
-$ export BRIDGE_KUBEADMIN_PASSWORD=$(cat "/path/to/install-dir/auth/kubeadmin-password")
-$ ./test-gui.sh e2e
-```
-
-If you don't want to run the entire e2e tests, you can use a different suite from [protractor.conf.ts](frontend/integration-tests/protractor.conf.ts). For instance,
-
-```
-$ ./test-gui.sh <suite>
-```
-
 ##### Hacking Protractor Tests
 
 To see what the tests are actually doing, it is posible to run in none `headless` mode by setting the `NO_HEADLESS` environment variable:
 
 ```
-$ NO_HEADLESS=true ./test-gui.sh <suite>
+$ NO_HEADLESS=true ./test-protractor.sh <suite>
 ```
 
 To use a specific binary version of chrome, it is posible to set the `CHROME_BINARY_PATH` environment variable:
 
 ```
-$ CHROME_BINARY_PATH="/usr/bin/chromium-browser" ./test-gui.sh <suite>
+$ CHROME_BINARY_PATH="/usr/bin/chromium-browser" ./test-protractor.sh <suite>
 ```
 
 To avoid skipping remaining portion of tests upon encountering the first failure, `NO_FAILFAST` environment variable can be used:
 
 ```
-$ NO_FAILFAST=true ./test-gui.sh <suite>
+$ NO_FAILFAST=true ./test-protractor.sh <suite>
 ```
 
 ##### Debugging Protractor Tests
 
 1. `cd frontend; yarn run build`
 2. Add `debugger;` statements to any e2e test
-3. `yarn run debug-test-suite --suite <suite-to-debug>`
+3. `yarn run debug-protractor-suite --suite <suite-to-debug>`
 4. Chrome browser URL: 'chrome://inspect/#devices', click on the 'inspect' link in **Target (v10...)** section.
 5. Launches chrome-dev tools, click Resume button to continue
 6. Will break on any `debugger;` statements
@@ -299,22 +294,30 @@ The end-to-end tests run against pull requests using [ci-operator](https://githu
 The tests are defined in [this manifest](https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/console/openshift-console-master-presubmits.yaml)
 in the [openshift/release](https://github.com/openshift/release) repo and were generated with [ci-operator-prowgen](https://github.com/openshift/ci-operator-prowgen).
 
-CI runs the [test-prow-e2e.sh](test-prow-e2e.sh) script, which runs the cypress tests and the protractor `e2e` test suite defined in [protractor.conf.ts](frontend/integration-tests/protractor.conf.ts).
+CI runs the [test-prow-e2e.sh](test-prow-e2e.sh) script, which runs [test-cypress.sh](test-cypress.sh) and ['test-protractor.sh e2e'](test-protractor.sh), which runs the protractor `e2e` test suite.
+
+##### Cypress in CI
+The CI executes [test-cypress.sh](test-cypress.sh) to run all Cypress tests, in all 'packages' (console, olm, and devconsole), in `-- headless` mode via:
+
+`test-cypress.sh -h true`
 
-You can simulate an e2e run against an existing 4.0 cluster with the following commands (replace `/path/to/install-dir` with your OpenShift 4.0 install directory):
+For more information on `test-cypress.sh` usage please see [Execute Cypress in different packages](#execute-cypress-in-different-packages)
+##### Protractor in CI
+['test-protractor.sh e2e'](test-protractor.sh) runs the protractor `e2e` test suite defined in [protractor.conf.ts](frontend/integration-tests/protractor.conf.ts)
+You can simulate an e2e run against an existing cluster with the following commands (replace `/path/to/install-dir` with your OpenShift install directory):
 
 ```
 $ oc apply -f ./frontend/integration-tests/data/htpasswd-secret.yaml
 $ oc patch oauths cluster --patch "$(cat ./frontend/integration-tests/data/patch-htpasswd.yaml)" --type=merge
 $ export BRIDGE_BASE_ADDRESS="$(oc get consoles.config.openshift.io cluster -o jsonpath='{.status.consoleURL}')"
 $ export BRIDGE_KUBEADMIN_PASSWORD=$(cat "/path/to/install-dir/auth/kubeadmin-password")
-$ ./test-gui.sh e2e
+$ ./test-protractor.sh e2e
 ```
 
 If you don't want to run the entire e2e tests, you can use a different suite from [protractor.conf.ts](frontend/integration-tests/protractor.conf.ts). For instance,
 
 ```
-$ ./test-gui.sh <suite>
+$ ./test-protractor.sh <suite>
 ```
 
 ### Deploying a Custom Image to an OpenShift Cluster

frontend/integration-tests/protractor.conf.ts

@@ -242,7 +242,7 @@ export const config = {
   params: {
     // Set to 'true' to enable OpenShift resources in the crud scenario.
     // Use a string rather than boolean so it can be specified on the command line:
-    // $ yarn test-gui --params.openshift true
+    // $ yarn test-protractor --params.openshift true
     openshift: 'false',
     // Set to 'true' to enable Service Catalog resources in the crud scenario.
     servicecatalog: 'false',

frontend/package.json

@@ -22,10 +22,11 @@
     "webdriver-update": "webdriver-manager update",
     "webdriver-update-macos": "CHROME_VERSION=$(/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome --version) && yarn webdriver-update --versions.chrome=\"${CHROME_VERSION}\"",
     "webdriver-update-fedora": "CHROME_VERSION=$(/usr/bin/google-chrome-stable --version) && yarn webdriver-update --versions.chrome=\"${CHROME_VERSION}\"",
-    "test-gui-tap": "TAP=true yarn test-gui",
-    "test-gui-openshift": "yarn test-suite --suite crud --params.openshift true",
-    "test-gui": "yarn test-suite --suite all",
-    "test-suite": "yarn ts-node ./node_modules/.bin/protractor integration-tests/protractor.conf.ts",
+    "test-protractor-tap": "TAP=true yarn test-protractor",
+    "test-protractor-openshift": "yarn test-protractor-suite --suite crud --params.openshift true",
+    "test-protractor": "yarn test-protractor-suite --suite all",
+    "test-protractor-suite": "yarn ts-node ./node_modules/.bin/protractor integration-tests/protractor.conf.ts",
+    "debug-protractor-suite": "TS_NODE_COMPILER_OPTIONS='{\"module\":\"commonjs\"}' node -r ts-node/register --inspect-brk ./node_modules/.bin/protractor integration-tests/protractor.conf.ts",
     "test-cypress-console": "cd packages/integration-tests-cypress && cypress open --env openshift=true",
     "test-cypress-olm": "cd packages/operator-lifecycle-manager/integration-tests-cypress && ../../../node_modules/.bin/cypress open --config-file cypress-olm.json --env openshift=true",
     "test-cypress-console-headless": "cd packages/integration-tests-cypress && node --max-old-space-size=4096 ../../node_modules/.bin/cypress run --env openshift=true --browser ${BRIDGE_E2E_BROWSER_NAME:=chrome} --headless",
@@ -34,7 +35,6 @@
     "cypress-generate": "marge -o ./gui_test_screenshots/ -f cypress-report -t 'OpenShift Console Cypress Test Results' -p 'OpenShift Cypress Test Results' --showPassed false --assetsDir ./gui_test_screenshots/cypress/assets ./gui_test_screenshots/cypress.json",
     "cypress-a11y-report": "echo '\nA11y Test Results:' && mv packages/integration-tests-cypress/cypress-a11y-report.json ./gui_test_screenshots/ && node -e \"console.table(JSON.parse(require('fs').readFileSync(process.argv[1])));\" ./gui_test_screenshots/cypress-a11y-report.json",
     "cypress-postreport": "yarn cypress-merge && yarn cypress-generate && yarn cypress-a11y-report",
-    "debug-test-suite": "TS_NODE_COMPILER_OPTIONS='{\"module\":\"commonjs\"}' node -r ts-node/register --inspect-brk ./node_modules/.bin/protractor integration-tests/protractor.conf.ts",
     "analyze": "NODE_ENV=production NODE_OPTIONS=--max-old-space-size=4096 yarn ts-node ./node_modules/.bin/webpack --mode=production --profile --json | sed '0,/^{/s/^[^{].*//g' > public/dist/stats.json && yarn ts-node ./node_modules/.bin/webpack-bundle-analyzer --mode static -r public/dist/report.html public/dist/stats.json",
     "prettier-all": "prettier --write '**/*.{js,jsx,ts,tsx,json}'",
     "ts-node": "ts-node -O '{\"module\":\"commonjs\"}'",

frontend/packages/operator-lifecycle-manager/src/components/descriptors/README.md

@@ -120,5 +120,5 @@ To sufficiently test your component, modify `descriptors.scenario.ts` and add a
 Run the OLM end-to-end tests against a cluster with OLM installed:
 
 ```shell
-$ yarn test-suite --suite olmFull
+$ yarn test-protractor-suite --suite olmFull
 ```

test-cypress.sh

@@ -35,9 +35,9 @@ if [ $# -eq 0 ]; then
 fi
 
 if [ -n "${headless-}" ] && [ -z "${pkg-}" ]; then
-  echo "yarn run test-cypress-console-headless"
-  echo "yarn run test-cypress-olm-headless"
-  echo "yarn run test-cypress-devconsole-headless"
+  yarn run test-cypress-console-headless
+  yarn run test-cypress-olm-headless
+  yarn run test-cypress-devconsole-headless
   exit;
 fi
 

test-gui.sh → test-protractor.sh

@@ -16,7 +16,7 @@ else
 fi
 
 if [ $# -gt 0 ] && [ -n "$1" ]; then
-  yarn run test-suite --suite "$1" --params.openshift true
+  yarn run test-protractor-suite --suite "$1" --params.openshift true
 else
-  yarn run test-gui --params.openshift true
+  yarn run test-protractor --params.openshift true
 fi

test-prow-e2e.sh

@@ -35,7 +35,7 @@ export DBUS_SESSION_BUS_ADDRESS
 SCENARIO="${1:-e2e}"
 
 if [ "$SCENARIO" != "login" ]; then
-  CHROME_VERSION=$(google-chrome --version) ./test-gui.sh "$SCENARIO"
+  CHROME_VERSION=$(google-chrome --version) ./test-protractor.sh "$SCENARIO"
 fi
 
 if [ "$SCENARIO" == "e2e" ] || [ "$SCENARIO" == "release" ]; then