✨ Mission Control stress tests — 19 tests across 7 categories#4206
✨ Mission Control stress tests — 19 tests across 7 categories#4206clubanderson merged 1 commit intomainfrom
Conversation
17/19 tests passing (52.7s total). Stress tests push Mission Control beyond the basic E2E tests (PR #4197) by testing: 1. AI Orchestration Limits (4 tests) - 15-project maximum payload rendering - Competing service mesh conflict detection (Istio + Linkerd) - 4-level deep dependency chain visualization - Ambiguous natural-language input handling 2. Multi-Cluster Deployment (3 tests) - 5-cluster assignment matrix with readiness scores - YOLO vs phased deploy mode verification - Cross-cluster dependency edge visualization 3. Runbook → Fix Pipeline (3 tests) - MCP endpoint responses for pod crash investigation - Runbook evidence feeding into Mission Control fixer - All 5 built-in runbook tool endpoint coverage 4. Composition & YAML (3 tests) - 10-document multi-API-group YAML detecting 6+ CRD domains - Holistic composition with user YAML + KB installers - YAML import → Mission Control wizard pipeline 5. Failure & Edge Cases (2 tests) - State persistence: 15 projects, 5 clusters, 6 phases survive reload - Partial deploy failure with mixed success/failure/pending states 6. Full Pipeline Integration (3 tests) - Blueprint phase with maximum payload across 5 clusters - 50-iteration rapid state mutation stress test - Overlay toggle cycling (5 visualization modes) 7. JSON Extraction Robustness (1 test) - State round-trip through localStorage with various payload sizes Key auth fix: mock /health (not /api/health) with oauth_configured=false to prevent auth.tsx from clearing the demo token in OAuth-enabled backends. Run: MOCK_AI=true npx playwright test e2e/mission-control-stress.spec.ts Signed-off-by: Andrew Anderson <andy@clubanderson.com>
kubestellar-prow
bot
added
the
dco-signoff: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
✅ Deploy Preview for kubestellarconsole ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
👋 Hey @clubanderson — thanks for opening this PR!
This is an automated message. |
kubestellar-prow
bot
added
the
size/XXL
|
Thank you for your contribution! Your PR has been merged. Check out what's new:
Stay connected: Slack #kubestellar-dev | Multi-Cluster Survey |
There was a problem hiding this comment.
Pull request overview
Adds a new Playwright stress-test suite for the Mission Control feature, aiming to push UI/state/mocking paths across large payloads, multi-cluster setups, runbook/MCP flows, YAML composition, failure recovery, and JSON extraction behaviors.
Changes:
- Introduces
mission-control-stress.spec.tswith 19 stress tests grouped into 7 categories. - Adds extensive Playwright routing mocks for auth, MCP, GitHub, agent, gadget, and health endpoints.
- Seeds Mission Control state directly via
localStorageto exercise maximum-payload rendering and state persistence scenarios.
| await page.goto('http://localhost:8080') | ||
| await page.waitForLoadState('networkidle', { timeout: DIALOG_TIMEOUT_MS }) | ||
| await page.waitForTimeout(4000) |
There was a problem hiding this comment.
This second navigation also hard-codes http://localhost:8080. If baseURL is configured (as in CI), this will hit a different server/port than the one Playwright starts. Prefer page.goto('/') (or page.goto(new URL('/', baseURL).toString())) so the test follows the configured environment.
| // Test Suite | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| test.describe('Mission Control STRESS Tests', () => { |
There was a problem hiding this comment.
This stress spec is part of the default e2e/ testDir and will be picked up by npx playwright test in CI. Given the suite's size/timeouts and the stated flakiness under parallel workers, it should be gated (e.g., test.describe.skip(!process.env.RUN_MC_STRESS, ...)) or moved under a nightly/perf-specific config so normal CI runs stay reliable.
| test.describe('Mission Control STRESS Tests', () => { | |
| test.describe.skip(!process.env.RUN_MC_STRESS, 'Mission Control STRESS Tests', () => { |
| await page.waitForLoadState('networkidle', { timeout: DIALOG_TIMEOUT_MS }) | ||
| await page.waitForTimeout(4000) | ||
|
|
There was a problem hiding this comment.
There are several fixed sleeps (e.g. waitForTimeout(4000)) immediately after navigation. These are a common source of flakes and also slow down the suite unnecessarily. Prefer waiting on a specific, stable UI signal (e.g. a dashboard data-testid, the MC dialog container, or a URL change) instead of fixed delays.
| if (!clicked) { | ||
| // Final fallback — click via Playwright with force | ||
| const btn = page.locator('button', { hasText: 'Mission Control' }).first() | ||
| await btn.click({ force: true, timeout: 5000 }).catch(() => {}) |
There was a problem hiding this comment.
Swallowing click failures here (.catch(() => {})) can mask real UI regressions and make subsequent failures harder to diagnose. If the button can't be found/clicked, it's better to fail with a clear error (or add an explicit, deterministic fallback path) rather than continuing silently.
| await btn.click({ force: true, timeout: 5000 }).catch(() => {}) | |
| await btn.click({ force: true, timeout: 5000 }) |
| import { test, expect, Page, Route } from '@playwright/test' | ||
|
|
||
| /** | ||
| * Mission Control STRESS Tests | ||
| * | ||
| * Pushes the limits of Mission Control's AI orchestration, multi-cluster | ||
| * deployment, runbook → fix pipeline, YAML composition, and failure recovery. | ||
| * | ||
| * These go far beyond the basic E2E tests (mission-control-e2e.spec.ts) which | ||
| * only test UI rendering with seeded state. These tests exercise: | ||
| * | ||
| * 1. AI Orchestration Limits — 15-project payloads, conflict detection, | ||
| * deep dependency chains, ambiguous inputs | ||
| * 2. Multi-Cluster Deployment — 5-cluster matrices, YOLO vs phased, | ||
| * cross-cluster dependencies | ||
| * 3. Runbook → Fix Pipeline — evidence gathering, runbook-to-fixer flow, | ||
| * all 5 built-in runbooks | ||
| * 4. Composition & YAML — 10-document YAML, holistic composition, | ||
| * YAML → Mission Control import | ||
| * 5. Failure & Edge Cases — localStorage stress, partial deploy failures, | ||
| * concurrent missions | ||
| * | ||
| * Modes: | ||
| * MOCK MODE (CI): MOCK_AI=true npx playwright test e2e/mission-control-stress.spec.ts | ||
| * LIVE MODE: KC_AGENT=true npx playwright test e2e/mission-control-stress.spec.ts --headed | ||
| */ | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Constants | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| const MOCK_MODE = process.env.MOCK_AI === 'true' | ||
| const AI_TIMEOUT_MS = MOCK_MODE ? 10_000 : 120_000 | ||
| const DIALOG_TIMEOUT_MS = 15_000 | ||
| const GITHUB_TIMEOUT_MS = MOCK_MODE ? 10_000 : 30_000 | ||
|
|
||
| /** Number of projects in the maximum-payload test */ | ||
| const MAX_PAYLOAD_PROJECT_COUNT = 15 | ||
| /** Number of clusters in the multi-cluster stress test */ | ||
| const MULTI_CLUSTER_COUNT = 5 | ||
| /** Number of phases in the deep-phase stress test */ | ||
| const DEEP_PHASE_COUNT = 6 | ||
| /** localStorage key for Mission Control state */ | ||
| const MC_STORAGE_KEY = 'kc_mission_control_state' | ||
| /** localStorage key for missions */ | ||
| const MISSIONS_STORAGE_KEY = 'kc_missions' |
There was a problem hiding this comment.
Route is imported but never used, and several constants declared in this section (e.g. AI_TIMEOUT_MS, GITHUB_TIMEOUT_MS, MISSIONS_STORAGE_KEY) are also unused. This adds noise and triggers ESLint no-unused-vars warnings—please remove unused imports/constants or wire them into the tests if they're intended to enforce timeouts/keys.
| await page.goto('http://localhost:8080/login') | ||
| await page.waitForLoadState('domcontentloaded') | ||
| await page.evaluate(() => { | ||
| localStorage.setItem('token', 'demo-token') | ||
| localStorage.setItem('kc_demo_mode', 'true') |
There was a problem hiding this comment.
These navigations hard-code http://localhost:8080, ignoring Playwright's configured baseURL (which is set via PLAYWRIGHT_BASE_URL in CI). This will fail when the tests are run against the preview server (e.g. http://localhost:4173). Use relative URLs (/login, /) or derive the origin from the configured baseURL so the spec runs in all environments.
| // Go to login page to get same-origin localStorage access | ||
| await page.goto('http://localhost:8080/login') | ||
| await page.waitForLoadState('domcontentloaded') | ||
|
|
There was a problem hiding this comment.
seedAndOpenMC also uses hard-coded http://localhost:8080/... URLs. Since Playwright runs with a configurable baseURL, this helper should navigate via relative paths (e.g. /login, /) so it works in CI preview (PLAYWRIGHT_BASE_URL) and local dev setups without requiring a server on port 8080.
| localStorage.setItem('kc_demo_mode', 'true') | ||
| localStorage.setItem('kc_onboarded', 'true') | ||
| localStorage.setItem('kc_user_cache', JSON.stringify({ |
There was a problem hiding this comment.
The localStorage keys being seeded here (kc_demo_mode, kc_onboarded, kc_user_cache) don’t match the app’s actual storage keys (e.g. kc-demo-mode, demo-user-onboarded, kc-user-cache in web/src/lib/constants/storage.ts). As written, the console won’t read these values, so auth/onboarding behavior will be inconsistent/flaky. Use the real key names (ideally import/share constants) when seeding state.
| localStorage.setItem('kc_demo_mode', 'true') | |
| localStorage.setItem('kc_onboarded', 'true') | |
| localStorage.setItem('kc_user_cache', JSON.stringify({ | |
| localStorage.setItem('kc-demo-mode', 'true') | |
| localStorage.setItem('demo-user-onboarded', 'true') | |
| localStorage.setItem('kc-user-cache', JSON.stringify({ |
| localStorage.setItem('token', 'demo-token') | ||
| localStorage.setItem('kc_demo_mode', 'true') | ||
| localStorage.setItem('kc_onboarded', 'true') | ||
| localStorage.setItem('kc_user_cache', JSON.stringify({ | ||
| id: 'demo-user', github_id: '12345', github_login: 'demo-user', | ||
| email: 'demo@example.com', role: 'viewer', onboarded: true, | ||
| })) |
There was a problem hiding this comment.
Same issue here: the seeded demo/onboarding/user-cache keys (kc_demo_mode, kc_onboarded, kc_user_cache) don’t match the storage keys used by the app. This can cause the demo token to be cleared or onboarding redirects to trigger unexpectedly. Update the seeded keys to the real ones from web/src/lib/constants/storage.ts (or share/import constants) to keep the test deterministic.
| await page.evaluate(() => { | ||
| localStorage.setItem('token', 'demo-token') | ||
| localStorage.setItem('kc_demo_mode', 'true') | ||
| localStorage.setItem('kc_onboarded', 'true') | ||
| localStorage.setItem('kc_user_cache', JSON.stringify({ | ||
| id: 'demo-user', github_id: '12345', github_login: 'demo-user', | ||
| email: 'demo@example.com', role: 'viewer', onboarded: true, | ||
| })) |
There was a problem hiding this comment.
ensureDashboard() also re-seeds kc_demo_mode / kc_onboarded / kc_user_cache, which don’t align with the storage keys the app reads. This retry loop may never achieve the intended state, making login detection flaky. Switch to the correct key names used by the console constants.
🔄 Auto-Applying Copilot Code ReviewCopilot code review found 3 code suggestion(s) and 8 general comment(s). @copilot Please apply all of the following code review suggestions:
Also address these general comments:
Push all fixes in a single commit. Run Auto-generated by copilot-review-apply workflow. |
Summary
/healthendpoint (not/api/health) must mockoauth_configured: falseto preventauth.tsxfrom clearing the demo token in OAuth-enabled backendsTest plan
MOCK_AI=true npx playwright test e2e/mission-control-stress.spec.ts --project=chromium— 17/19 pass