From 83aa06f96906724efc4a84382656402e09059ea6 Mon Sep 17 00:00:00 2001 From: Evelyn Tanigawa Murasaki Date: Tue, 18 Nov 2025 08:58:55 -0300 Subject: [PATCH] monitoring-plugin testing instructions - still manual --- AGENTS.md | 40 ++- CLAUDE.md | 1 + web/cypress/CYPRESS_TESTING_GUIDE.md | 254 ++++++++++++++++++ web/cypress/E2E_TEST_SCENARIOS.md | 19 +- web/cypress/README.md | 378 +++++++++++++++++++-------- web/cypress/configure-env.sh | 4 +- 6 files changed, 574 insertions(+), 122 deletions(-) create mode 100644 CLAUDE.md create mode 100644 web/cypress/CYPRESS_TESTING_GUIDE.md diff --git a/AGENTS.md b/AGENTS.md index 067f8133..12cc6902 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -113,7 +113,7 @@ make lint-frontend make lint-backend make test-translations make test-backend -# Run cypress tests (see web/cypress/README.md) +# future slash command for test execution ``` ### PR Requirements: @@ -121,6 +121,44 @@ make test-backend - **Testing**: All linting and tests must pass - **Translations**: Ensure i18next keys are properly added +### Cypress E2E Testing + +#### Overview +The Monitoring Plugin uses Cypress for comprehensive End-to-End (E2E) testing to ensure functionality across both the core **monitoring-plugin** (managed by CMO) and the **monitoring-console-plugin** (managed by COO). Our test suite covers test scenarios including alerts, metrics, dashboards, and integration with Virtualization and Fleet Management (ACM). + +**Key Testing Documentation:** +- **Setup & Configuration**: `web/cypress/README.md` - Environment variables, installation, troubleshooting +- **Testing Guide**: `web/cypress/CYPRESS_TESTING_GUIDE.md` - Test architecture, creating tests, workflows +- **Test Catalog**: `web/cypress/E2E_TEST_SCENARIOS.md` - Complete list of all test scenarios + +#### When to Create New Cypress Tests + +You should create new Cypress tests when: + +1. **Adding New Features**: Any new UI feature requires corresponding E2E tests +2. **Fixing Bugs**: Bug fixes should include tests to prevent regression +3. **Modifying Existing Features**: Changes to existing functionality require test updates + +#### Quick Test Commands + +```bash +cd web/cypress + +# Run all regression tests +npm run cypress:run --spec "cypress/e2e/**/regression/**" + +# Run BVT (Build Verification Tests) +npm run cypress:run --spec "cypress/e2e/monitoring/00.bvt_admin.cy.ts" + +# Run COO tests +npm run cypress:run --spec "cypress/e2e/coo/*.cy.ts" + +# Interactive mode +npm run cypress:open +``` + +For detailed testing instructions, see `web/cypress/CYPRESS_TESTING_GUIDE.md` + ### Release Pipeline: - **Konflux**: Handles CI/CD and release automation - **CMO releases**: Follow OpenShift release cycles diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..47dc3e3d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/web/cypress/CYPRESS_TESTING_GUIDE.md b/web/cypress/CYPRESS_TESTING_GUIDE.md new file mode 100644 index 00000000..91d9223f --- /dev/null +++ b/web/cypress/CYPRESS_TESTING_GUIDE.md @@ -0,0 +1,254 @@ +# Cypress Testing Guide - Monitoring Plugin + +> **Complete guide for developers and AI agents on Cypress E2E testing** + +--- + +## Table of Contents +- [Quick Start](#quick-start) +- [Test Architecture](#test-architecture) +- [Creating Tests](#creating-tests) +- [Running Tests](#running-tests) +- [Troubleshooting](#troubleshooting) + +--- + +## Quick Start + +### Prerequisites +- Node.js >= 18 +- OpenShift cluster with kubeconfig +- Environment variables configured + +### 30-Second Setup +```bash +cd web/cypress +source ./configure-env.sh # Interactive configuration +npm install # Install dependencies +npm run cypress:open # Start testing +``` + +**For detailed setup instructions and environment configuration, see [README.md](README.md)** + +--- + +## Test Architecture + +### 3-Layer Organization + +The Monitoring Plugin uses a 3-layer architecture for test organization: + +``` +┌─────────────────────────────────────────────────┐ +│ Layer 3: E2E Test Files │ +│ (cypress/e2e/) │ +│ - Call support scenarios │ +│ - Specify perspective (Administrator, etc.) │ +└────────────────┬────────────────────────────────┘ + │ imports +┌────────────────▼────────────────────────────────┐ +│ Layer 2: Support Scenarios │ +│ (cypress/support/monitoring or perses │ +│ - Reusable test scenarios │ +│ - Work across multiple perspectives │ +│ - Export functions with perspective parameter │ +└────────────────┬────────────────────────────────┘ + │ uses +┌────────────────▼────────────────────────────────┐ +│ Layer 1: Page Object Views │ +│ (cypress/views/) │ +│ - Reusable UI actions │ +│ - Navigation, clicks, assertions │ +│ - Use data-test attributes │ +└─────────────────────────────────────────────────┘ +``` + +### File Structure + +``` +cypress/ +├── e2e/ +│ ├── monitoring/ # Core monitoring tests (Administrator) +│ │ ├── 00.bvt_admin.cy.ts +│ │ └── regression/ +│ ├── coo/ # COO-specific tests +│ │ ├── 01.coo_bvt.cy.ts +│ │ └── 02.acm_alerting_ui.cy.ts +│ └── virtualization/ # Integration tests (Virtualization) +├── support/ +│ ├── monitoring/ # Reusable test scenarios +│ │ ├── 01.reg_alerts.cy.ts +│ │ ├── 02.reg_metrics.cy.ts +│ │ └── 03.reg_legacy_dashboards.cy.ts +│ ├── perses/ # COO/Perses scenarios +│ └── commands/ # Custom Cypress commands +└── views/ # Page object models (reusable actions) +``` + +**Benefits**: +- Test scenarios reusable across Administrator, Virtualization, and Fleet Management perspectives +- Page actions separated from test logic for better maintainability +- UI changes only require updating views, not individual tests + +--- + +## Creating Tests + +### Workflow + +1. **Layer 1 - Views**: Check/add page actions in `cypress/views/` + - Under `views/` folder, find pre-defined actions per page + - If none fits your needs, add new ones + +2. **Layer 2 - Support**: Add test scenarios to `cypress/support/monitoring/` + - Add test scenarios to cypress files under `support/` folder + - Make scenarios reusable across perspectives (Administrator, Virtualization, Fleet Management) + - If it is not applicable, in some cases for Incidents or Fleet Management, test scenarios will be written directly into Layer 3 + +3. **Layer 3 - E2E**: Verify e2e files call your scenario (usually pre-configured) + - Administrator: `e2e/monitoring/` + - Virtualization: `e2e/virtualization/` + - Fleet Management: `e2e/coo/` (for ACM) + +### Example: Support Scenario Structure + +```typescript +// In support/monitoring/01.reg_alerts.cy.ts +import { nav } from '../../views/nav'; +import { silencesListPage } from '../../views/silences-list-page'; + +export const runAlertTests = (perspective: string) => { + describe(`${perspective} perspective - Alerting > Alerts page`, () => { + + it('should filter alerts by severity', () => { + // Use page object actions from views/ + silencesListPage.filter.byName('test-alert'); + silencesListPage.rows.shouldBe('test-alert', 'Active'); + }); + }); +}; +``` + +### When to Create New Tests + +| Scenario | Action | +|----------|--------| +| New UI feature | Create new test scenario in support/ | +| Bug fix | Add test case to existing support file | +| Component update | Update existing test scenarios | +| New Perses feature | Create new test scenario in support/ | +| ACM integration | Add test in e2e/coo/ | + +### Best Practices + +1. **Use Page Objects**: Import actions from `cypress/views/` +2. **Data Test Attributes**: Prefer `data-test` over CSS selectors +3. **Keep Tests Isolated**: Each test should run independently +4. **Meaningful Assertions**: Use descriptive error messages +5. **Document Changes**: Update `E2E_TEST_SCENARIOS.md` + +--- + +## Running Tests + +### Common Commands + +```bash +cd web/cypress + +# Run all regression tests +npm run cypress:run --spec "cypress/e2e/**/regression/**" + +# Run specific feature regression +npm run cypress:run --spec "cypress/e2e/monitoring/regression/01.reg_alerts_admin.cy.ts" +npm run cypress:run --spec "cypress/e2e/monitoring/regression/02.reg_metrics_admin.cy.ts" +npm run cypress:run --spec "cypress/e2e/monitoring/regression/03.reg_legacy_dashboards_admin.cy.ts" + +# Run BVT (Build Verification Tests) +npm run cypress:run --spec "cypress/e2e/monitoring/00.bvt_admin.cy.ts" + +# Run COO tests +npm run cypress:run --spec "cypress/e2e/coo/01.coo_bvt.cy.ts" + +# Run ACM Alerting tests +npm run cypress:run --spec "cypress/e2e/coo/02.acm_alerting_ui.cy.ts" + +# Interactive mode (GUI) +npm run cypress:open +``` + +### Environment Setup + +**Interactive** (Recommended): +```bash +cd web/cypress +source ./configure-env.sh +``` + +**Manual Setup**: For complete environment variable reference and configuration examples, see [README.md](README.md#environment-variables-reference) + +### Regression Testing Strategy + +| Change Type | Required Tests | +|-------------|---------------| +| **UI Component Change** | Feature-specific regression + BVT | +| **API Integration Change** | Full regression suite | +| **Console Extension Change** | BVT + Navigation tests | +| **Bug Fix** | New test + Related regression | + +### Pre-PR Checklist + +- [ ] `make lint-frontend` (no errors) +- [ ] `make lint-backend` (no errors) +- [ ] Ran BVT tests locally (all passing) +- [ ] Ran regression tests for affected features (all passing) +- [ ] Created/updated tests for new features or bug fixes +- [ ] Updated `E2E_TEST_SCENARIOS.md` if added new tests + +--- + +## Troubleshooting + +### Debugging Failed Tests + +1. **Check test videos**: `web/cypress/videos/` +2. **Check screenshots**: `web/cypress/screenshots/` +3. **Run with debug**: + ```bash + export CYPRESS_DEBUG=true + npm run cypress:run + ``` +4. **Run interactively**: + ```bash + npm run cypress:open + ``` + +### Common Test Issues + +| Issue | Solution | +|-------|----------| +| Test fails intermittently | Check for timing issues, add proper waits | +| Element not found | Verify data-test attributes exist, check page object | +| Assertion fails | Review expected vs actual values, update test | +| Test hangs | Check for infinite loops or missing assertions | + +### Setup & Configuration Issues + +For environment variable issues, login problems, kubeconfig errors, and installation troubleshooting, see [README.md](README.md#troubleshooting-setup-issues) + +### CI/CD Integration + +Cypress tests run automatically in the CI pipeline: +- **Pre-merge**: BVT tests run on every PR +- **Post-merge**: Full regression suite runs on main branch +- **Konflux Pipeline**: Automated testing for release candidates + +--- + +## Additional Resources + +- **Cypress Documentation**: https://docs.cypress.io/ +- **Test Scenarios Catalog**: `E2E_TEST_SCENARIOS.md` +- **Setup Instructions**: `README.md` +- **Main Guide**: `../../AGENTS.md` + diff --git a/web/cypress/E2E_TEST_SCENARIOS.md b/web/cypress/E2E_TEST_SCENARIOS.md index 7dc05a87..be80f9a7 100644 --- a/web/cypress/E2E_TEST_SCENARIOS.md +++ b/web/cypress/E2E_TEST_SCENARIOS.md @@ -20,6 +20,12 @@ Located in `e2e/coo/` |------|------------|---------------|-------------| | `01.coo_bvt.cy.ts` | BVT: COO | 1. Admin perspective - Observe Menu | Verifies Observe menu navigation and submenus (Alerting, Silences, Alerting rules, Dashboards (Perses)) | +### ACM Alerting UI Tests + +| File | Test Suite | Test Scenario | Description | +|------|------------|---------------|-------------| +| `02.acm_alerting_ui.cy.ts` | ACM Alerting UI | 1. Fleet Management perspective - ACM Alerting | Validates ACM integration with COO, Fleet Management perspective navigation, local-cluster access, and ACM alert visibility (Watchdog, Watchdog-spoke, ClusterCPUHealth) | + --- ## Virtualization Tests @@ -175,22 +181,11 @@ These test scenarios are reusable test suites called by the main E2E test files. --- -## Test Statistics Summary - -| Category | Test Files | Direct it() Scenarios | Support Module Scenarios | Total Scenarios | -|----------|------------|----------------------|-------------------------|-----------------| -| **COO Tests** | 1 | 1 | 0 | 1 | -| **Virtualization Tests** | 4 | 6 | ~30+ (via support modules) | ~36+ | -| **Monitoring Tests** | 4 | 9 | ~30+ (via support modules) | ~39+ | -| **Support Modules** | 8 | 0 | 39 | 39 | -| **TOTAL** | **20** | **29** | **39** | **~128+** | - ---- - ## Perspectives Tested - **Administrator** - Standard admin perspective with full cluster access - **Virtualization** - Virtualization-specific perspective with integrated monitoring +- **Fleet Management** - ACM multi-cluster management perspective with observability integration ## Namespace Scopes diff --git a/web/cypress/README.md b/web/cypress/README.md index 4ed4c12c..b68e4f6b 100644 --- a/web/cypress/README.md +++ b/web/cypress/README.md @@ -1,178 +1,340 @@ -# Openshift Monitoring Plugin and Monitoring Console Plugin UI Tests -These console tests are related to Monitoring Plugin deployed by Cluster Monitoring Operator (CMO) as part of OCP - Observe menu with Alerting, Metrics, Dashboards pages and other related Alerting links. -Besides, Monitoring Console Plugin deployed by Cluster Observability Operator through Monitoring UIPlugin installation. +# Cypress Setup & Configuration Guide -## Test Documentation -For a comprehensive overview of all E2E test scenarios, including COO (Cluster Observability Operator), Monitoring, and Incidents tests, see [E2E_TEST_SCENARIOS.md](./E2E_TEST_SCENARIOS.md). +> **Technical setup and environment configuration for Monitoring Plugin Cypress tests** -## Prerequisite -1. [node.js](https://nodejs.org/) >= 18 +For testing workflows, test architecture, and creating tests, see **[CYPRESS_TESTING_GUIDE.md](CYPRESS_TESTING_GUIDE.md)** +--- + +## Quick Start -## Install dependencies -All required dependencies are defined in `package.json` in order to run Cypress tests, run `npm install` so that dependencies will be installed in `node_modules` folder ```bash -$ npm install -$ ls -ltr -node_modules/ -> dependencies will be installed at runtime here +cd web/cypress +npm install # Install dependencies +source ./configure-env.sh # Interactive configuration +npm run cypress:open # Start Cypress GUI ``` -## Running locally +--- -### Export necessary variables -in order to run Cypress tests, we need to export some environment variables that Cypress can read then pass down to our tests, currently we have following environment variables defined and used. +## Prerequisites -Using a non-admin user. -```bash -export CYPRESS_BASE_URL=https:// -export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider -export CYPRESS_LOGIN_USERS=username:password -export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig -``` -Using kubeadmin user. -```bash -export CYPRESS_BASE_URL=https:// -export CYPRESS_LOGIN_IDP=kube:admin -export CYPRESS_LOGIN_USERS=kubeadmin:password -export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig -``` -Set the following var to use custom Monitoring Plugin image (that goes on Cluster Monitoring Operator). The image will be patched in CMO CSV. -```bash -export CYPRESS_MP_IMAGE= -``` +- **Node.js**: >= 18 + +--- + +## Installation + +Install Cypress and all dependencies: -Set the var to skip Cluster Observability and all the required operators installation. ```bash -export CYPRESS_SKIP_COO_INSTALL=true +npm install ``` -Set the var to install Cluster Observability Operator from redhat-operators catalog source. +Dependencies are defined in `package.json` and will be installed in `node_modules/`. + +--- + +## Environment Configuration + +### Interactive Setup (Recommended) + +The `configure-env.sh` script provides an interactive way to set up all required environment variables: + ```bash -export CYPRESS_COO_UI_INSTALL=true +source ./configure-env.sh ``` -Set the var to install Cluster Observability Operator using Konflux bundle. +**Features**: +- Automatic prompting for all CYPRESS_ variables +- Automatic discovery and numbered selection of `*kubeconfig*` files in `$HOME/Downloads` +- Validates required variables + +**Alternative - Generate Export File**: ```bash -export CYPRESS_KONFLUX_COO_BUNDLE_IMAGE= +./configure-env.sh ``` -Set the var to use custom Cluster Observability Operator bundle image. +Creates `export-env.sh` that you can source later: `source export-env.sh` + +--- + +## Environment Variables Reference + +### Required Variables + +| Variable | Description | Example | +|----------|-------------|---------| +| `CYPRESS_BASE_URL` | OpenShift Console URL | `https://console-openshift-console.apps...` | +| `CYPRESS_LOGIN_IDP` | Identity provider name | `flexy-htpasswd-provider` or `kube:admin` | +| `CYPRESS_LOGIN_USERS` | Login credentials | `username:password` or `kubeadmin:password` | +| `CYPRESS_KUBECONFIG_PATH` | Path to kubeconfig file | `~/Downloads/kubeconfig` | + +### Plugin Image Configuration + +| Variable | Description | Use Case | +|----------|-------------|----------| +| `CYPRESS_MP_IMAGE` | Custom Monitoring Plugin image | Testing custom MP builds | +| `CYPRESS_MCP_CONSOLE_IMAGE` | Custom Monitoring Console Plugin image | Testing custom MCP builds | + +### Operator Installation Control + +| Variable | Default | Description | +|----------|---------|-------------| +| `CYPRESS_SKIP_COO_INSTALL` | `false` | Skip Cluster Observability Operator installation | +| `CYPRESS_SKIP_KBV_INSTALL` | `false` | Skip OpenShift Virtualization installation | +| `CYPRESS_SKIP_ALL_INSTALL` | `false` | Skip all operator installations (for pre-provisioned clusters) | +| `CYPRESS_COO_UI_INSTALL` | `false` | Install COO from redhat-operators catalog | +| `CYPRESS_KBV_UI_INSTALL` | `false` | Install Virtualization from redhat-operators catalog | + +### Bundle Images + +| Variable | Description | +|----------|-------------| +| `CYPRESS_KONFLUX_COO_BUNDLE_IMAGE` | COO bundle image from Konflux | +| `CYPRESS_CUSTOM_COO_BUNDLE_IMAGE` | Custom COO bundle image | +| `CYPRESS_KONFLUX_KBV_BUNDLE_IMAGE` | Virtualization bundle image from Konflux | +| `CYPRESS_CUSTOM_KBV_BUNDLE_IMAGE` | Custom Virtualization bundle image | + +### FBC images + +| Variable | Description | +|----------|-------------| +| `CYPRESS_FBC_STAGE_COO_IMAGE` | Cluster Observability Operator FBC image | +| `CYPRESS_FBC_STAGE_KBV_IMAGE` | Virtualization FBC image | + +### Testing Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `CYPRESS_SESSION` | `false` | Enable session management for faster execution | +| `CYPRESS_DEBUG` | `false` | Enable debug mode logging in headless mode | + +### Incidents Testing Configuration + +**Used primarily for Incidents feature testing:** + +| Variable | Default | Description | +|----------|---------|-------------| +| `CYPRESS_TIMEZONE` | `UTC` | Cluster timezone for incident timeline calculations | +| `CYPRESS_MOCK_NEW_METRICS` | `false` | Transform old metric names to new format in mocks (temporary workaround for testing against locally built instances) | + +**Example:** ```bash -export CYPRESS_CUSTOM_COO_BUNDLE_IMAGE= +export CYPRESS_TIMEZONE="America/New_York" +export CYPRESS_MOCK_NEW_METRICS=true ``` -Set the following var to use custom Monitoring Console Plugin UI plugin image. The image will be patched in Cluster Observability Operator CSV. +--- + +## Configuration Examples + +### Example 1: Testing with Non-Admin User + ```bash -export CYPRESS_MCP_CONSOLE_IMAGE= +export CYPRESS_BASE_URL=https://console-openshift-console.apps.cluster.example.com +export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider +export CYPRESS_LOGIN_USERS=testuser:testpassword +export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig ``` -Set the following var to specify the cluster timezone for incident timeline calculations. Defaults to UTC if not specified. +### Example 2: Testing with Kubeadmin + ```bash -export CYPRESS_TIMEZONE= +export CYPRESS_BASE_URL=https://console-openshift-console.apps.cluster.example.com +export CYPRESS_LOGIN_IDP=kube:admin +export CYPRESS_LOGIN_USERS=kubeadmin:admin-password +export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig ``` -Set the following var to transform old metric names to new format in mocks (temporary workaround for testing against locally built instances). +### Example 3: Testing Custom Plugin Build + ```bash -export CYPRESS_MOCK_NEW_METRICS=false +# Required variables +export CYPRESS_BASE_URL=https://... +export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider +export CYPRESS_LOGIN_USERS=username:password +export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig + +# Custom image +export CYPRESS_MP_IMAGE=quay.io/myorg/monitoring-plugin:my-branch +export CYPRESS_MCP_CONSOLE_IMAGE=quay.io/myorg/monitoring-console-plugin:my-branch ``` -Set the following var to enable Cypress session management for faster test execution. +### Example 4: Pre-Provisioned Cluster (Skip Installations) + ```bash -export CYPRESS_SESSION=true +# Required variables +export CYPRESS_BASE_URL=https://... +export CYPRESS_LOGIN_IDP=flexy-htpasswd-provider +export CYPRESS_LOGIN_USERS=username:password +export CYPRESS_KUBECONFIG_PATH=~/Downloads/kubeconfig + +# Skip installations (cluster already configured) +export CYPRESS_SKIP_ALL_INSTALL=true ``` -Set the following var to enable Cypress debug mode to log in headless mode. +### Example 5: Debug Mode + ```bash +# Required variables + debug export CYPRESS_DEBUG=true +export CYPRESS_SESSION=true # Faster test execution ``` -Set the following var to skip all operator installation, cleanup, and verifications (useful for pre-provisioned environments where COO and Monitoring UI Plugin are already installed). -```bash -export CYPRESS_SKIP_ALL_INSTALL=false -``` +--- -Integration Testing variables +## Running Cypress -Set the var to skip Openshift Virtualization and all the required operators installation. -```bash -export CYPRESS_SKIP_KBV_INSTALL=false -``` +### Interactive Mode (GUI) -Set the var to install Openshift Virtualization from redhat-operators catalog source. -```bash -export CYPRESS_KBV_UI_INSTALL=true -``` +Best for test development and debugging: -Set the var to install Openshift Virtualization Operator using Konflux bundle. ```bash -export CYPRESS_KONFLUX_KBV_BUNDLE_IMAGE= +npm run cypress:open ``` -# Set the var to use custom Openshift Virtualization Operator bundle image +### Headless Mode (CI-style) + +For automated testing: + ```bash -export CYPRESS_CUSTOM_KBV_BUNDLE_IMAGE= +npm run cypress:run ``` -Set the var to use Openshift Virtualization Operator FBC image +### Running Specific Tests + ```bash -export CYPRESS_FBC_STAGE_KBV_IMAGE= -``` +# COO BVT tests +npm run cypress:run --spec "cypress/e2e/coo/01.coo_bvt.cy.ts" -### Environment Configuration Script +# ACM Alerting tests +npm run cypress:run --spec "cypress/e2e/coo/02.acm_alerting_ui.cy.ts" -The `configure-env.sh` script provides an interactive way to set up all the required environment variables. This script eliminates the need to manually export each variable and helps find the correct kubeconfig file. +# Monitoring BVT tests +npm run cypress:run --spec "cypress/e2e/monitoring/00.bvt_admin.cy.ts" -**Features:** -- Automatic prompting for all CYPRESS_ variables -- Automatic discovery and numbered selection of `*kubeconfig*` files in `$HOME/Downloads` dir +# All Monitoring Regression tests +npm run cypress:run --spec "cypress/e2e/monitoring/regression/**" -**Usage:** -```bash -# Note: source command requires Bash shell -source ./configure-env.sh -``` -To export variables directly (Bash only). +# All Virtualization IVT tests +npm run cypress:run --spec "cypress/e2e/virtualization/**" -**File generation** -```bash -./configure-env.sh +# Incidents tests (requires CYPRESS_TIMEZONE and optionally CYPRESS_MOCK_NEW_METRICS) +npm run cypress:run --spec "cypress/e2e/**/incidents*.cy.ts" ``` -Creates an export file you can source later. (`source "export-env.sh`) +**Note**: Incidents tests require `CYPRESS_TIMEZONE` to be set to match your cluster's timezone configuration. See [Incidents Testing Configuration](#incidents-testing-configuration) for details. -### Before running cypress -- Make sure cluster's kubeconfig file is located at the correct environment variable / path you have exported -- The file to run Monitoring Plugin tests: bvt.cy.ts -- The file to run Monitoring Console Plugin tests (COO with Monitoring UIPlugin): coo_bvt.cy.ts +**For comprehensive test commands and regression testing strategies, see [CYPRESS_TESTING_GUIDE.md](CYPRESS_TESTING_GUIDE.md)** -### Start Cypress -We can either open Cypress GUI(open) or run Cypress in headless mode(run) to run the tests. -```bash -npx cypress open -npx cypress run -``` +--- + +## Test Results + +### Videos + +Test recordings are saved automatically: +- **Location**: `web/cypress/videos/` +- **Format**: `.mp4` +- **Generated**: For all test runs (pass or fail) + +### Screenshots -Some examples to run a specific file(s) +Screenshots captured on test failures: +- **Location**: `web/cypress/screenshots/` +- **Format**: `.png` +- **Generated**: Only on failures -It runs the COO BVT only +--- + +## Troubleshooting Setup Issues + +### Issue: Cypress Cannot Find Chrome/Browser + +**Solution**: Install Chrome or specify browser ```bash -cd monitoring-plugin/web/cypress -npx cypress run --spec "cypress/e2e/coo/01.coo_bvt.cy.ts" +npm run cypress:open --browser firefox ``` -It runs the Monitoring BVT only +### Issue: Environment Variables Not Set + +**Symptoms**: Tests fail with "BASE_URL is not defined" + +**Solution**: +1. Verify variables are exported: `echo $CYPRESS_BASE_URL` +2. Re-run configuration: `source ./configure-env.sh` +3. Ensure you're in the correct shell session + +### Issue: Kubeconfig Not Found + +**Symptoms**: "ENOENT: no such file or directory" + +**Solution**: ```bash -npx cypress run --spec "cypress/e2e/monitoring/01.bvt_monitoring.cy.ts" +# Check file exists +ls -la $CYPRESS_KUBECONFIG_PATH + +# Update path if needed +export CYPRESS_KUBECONFIG_PATH=/correct/path/to/kubeconfig ``` -It runs the Monitoring Regression tests +### Issue: Login Fails + +**Symptoms**: "User authentication failed" + +**Solution**: +1. Verify IDP name: Check OpenShift OAuth configuration +2. Verify credentials are correct +3. For kubeadmin, use `kube:admin` as IDP + +### Issue: Tests Are Slow + +**Solution**: Enable session management ```bash -npx cypress run --spec "cypress/e2e/monitoring/regression/**" +export CYPRESS_SESSION=true ``` -It runs the Virtualization IVT tests -```bash -npx cypress run --spec "cypress/e2e/virtualization/**" +--- + +## Test Organization + +### Directory Structure + ``` +cypress/ +├── e2e/ # Test files by perspective +│ ├── monitoring/ # Core monitoring (Administrator) +│ ├── coo/ # COO-specific tests +│ └── virtualization/ # Virtualization integration +├── support/ # Reusable test scenarios +│ ├── monitoring/ # Test scenario modules +│ ├── perses/ # Perses scenarios +│ └── commands/ # Custom Cypress commands +├── views/ # Page object models +├── fixtures/ # Test data and mocks +└── E2E_TEST_SCENARIOS.md # Complete test catalog +``` + +**For test architecture and creating new tests, see [CYPRESS_TESTING_GUIDE.md](CYPRESS_TESTING_GUIDE.md)** + +--- + +## Documentation + +- **Testing Guide**: [CYPRESS_TESTING_GUIDE.md](CYPRESS_TESTING_GUIDE.md) - Complete testing workflows and test creation +- **Test Scenarios**: [E2E_TEST_SCENARIOS.md](./E2E_TEST_SCENARIOS.md) - Catalog of all test scenarios +- **Project Guide**: [AGENTS.md](../../AGENTS.md) - Main developer guide +- **Cypress Docs**: https://docs.cypress.io/ - Official Cypress documentation + +--- + +## Additional Resources + +- **Configure Script**: `./configure-env.sh` - Interactive setup +- **Export Script**: `./export-env.sh` - Generated environment file +- **Fixtures**: `./fixtures/` - Test data and mocks +- **Support**: `./support/` - Custom commands and utilities + +--- -### Testing recording -You can access the recording for your test under monitoring-plugin/web/cypress/videos folder \ No newline at end of file +*For questions about test architecture, creating tests, or testing workflows, refer to [CYPRESS_TESTING_GUIDE.md](CYPRESS_TESTING_GUIDE.md)* diff --git a/web/cypress/configure-env.sh b/web/cypress/configure-env.sh index 86bd7b27..2ec31561 100755 --- a/web/cypress/configure-env.sh +++ b/web/cypress/configure-env.sh @@ -99,7 +99,9 @@ ask_yes_no() { bool_to_default_yn() { # Map truthy/falsey env values to y/n default for yes/no prompts local v=${1-} - case "${v,,}" in + # Convert to lowercase in a portable way + v=$(echo "$v" | tr '[:upper:]' '[:lower:]') + case "$v" in true|1|yes|y) echo "y" ;; false|0|no|n|"") echo "n" ;; *) echo "n" ;;