Skip to content

Improve Playwright reliability checks (codex) - #2372

Merged
numnx merged 1 commit into
feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3from
task/feature-codux-143-qs-coverage-observability-and--t08-codex-mr9l88tm
Jul 6, 2026
Merged

Improve Playwright reliability checks (codex)#2372
numnx merged 1 commit into
feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3from
task/feature-codux-143-qs-coverage-observability-and--t08-codex-mr9l88tm

Conversation

@numnx

@numnx numnx commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

🧩 Task: Improve Playwright reliability checks

Automated task PR opened by Code UX for T08.

📋 Summary

Sprint: Sprint 143 · QS: Coverage Observability and CI Reliability Audit · Task: T08

Strengthen Code UX coverage observability and CI reliability through enforceable tests, workflow hardening, structured observability guarantees, security validation, and aligned operational documentation without lowering coverage thresholds.

📝 Full Task Prompt
## Objective
Make Playwright E2E reliability enforceable across clean CI and local runs by validating compiled server startup, `/health` liveness, isolated state, viewport coverage, and artifact behavior.

## Scope
- `playwright.config.ts`
- `tests/e2e/helpers/prepare-app.ts`
- `tests/e2e/example.spec.ts`
- `tests/e2e/accessibility-smoke.spec.ts`
- `tests/e2e/app-release-smoke.spec.ts`
- `tests/e2e/project-setup-release.spec.ts`
- `.github/workflows/playwright.yml`
- `docs/development/testing-and-quality.md`
- `docs/operations/runbook.md`
- Current verification gap: Playwright uses `/health` and temp home, but the audit needs explicit E2E assertions for startup readiness, no fixed local state, viewport coverage, browser cache assumptions, and artifact retention.
- Desired guarantee: E2E runs only against the local compiled app, isolates HOME/USERPROFILE, avoids fixed developer state, checks desktop/mobile viewports where relevant, and uploads useful traces/reports on failure.

## Implementation Requirements
1. Add Playwright tests or helper assertions that call `/health` through `request`, verify the app shell loads from `baseURL`, and never navigate to external websites.
2. Extend fixture helpers to make run IDs deterministic enough for retries while still isolating worker/retry state; assert project fixture names and paths are unique per worker/retry.
3. Add a mobile viewport check to an existing responsive or accessibility spec using Playwright viewport APIs without enabling broad parallelism.
4. Review `playwright.config.ts` artifact settings and ensure trace, screenshot, video, output directory, reporter, `reuseExistingServer: false`, and temp `HOME`/`USERPROFILE` behavior are covered by tests or documented assertions.
5. Harden `.github/workflows/playwright.yml` if needed so build precedes E2E, browser install cannot rely only on cache, artifacts upload on `always()`, and Windows avoids stale `node_modules` cache assumptions.
6. Update `docs/development/testing-and-quality.md` and `docs/operations/runbook.md` with the Playwright startup, health, isolation, viewport, and artifact guarantees.

## Constraints
- Do not require provider credentials, Docker provider startup, worker dispatch, sprint execution, or real project state in E2E tests.
- Keep Playwright `workers: 1` and avoid fixed sleeps or browser-cache-dependent assertions.
- Do not switch readiness from `/health` to `/ready` for clean CI startup.
- Preserve cross-platform behavior for Linux, macOS, and Windows runners.

## Verification
- `pnpm run build`
- `pnpm exec playwright test tests/e2e/example.spec.ts tests/e2e/accessibility-smoke.spec.ts tests/e2e/app-release-smoke.spec.ts tests/e2e/project-setup-release.spec.ts`
- `pnpm exec playwright test`
- `pnpm run lint`
- Success means Playwright starts the compiled server from a clean temp home, passes local health and viewport checks, and CI artifact/cache behavior is documented and reliable.

✅ QA Review

⏳ QA review runs after this PR is opened. Check the task card in the dashboard for the outcome once it lands.

🤖 Model & Provider

Provider Codex CLI
Model gpt-5.5

⏱️ Timing

Started 2026-07-06 19:03:04 UTC
Finished 2026-07-06 19:18:09 UTC
Duration 15m 5s

📊 Token Usage

Metric Value
Input tokens 207,521
Cached input tokens 2,476,672
Output tokens 14,626
Total tokens 2,698,819
Invocations 1
Tool calls 57
Active time 13m 19s

Included usage estimate (subscription/local login): $2.71

🌿 Branch Info

Base: feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3
Head: task/feature-codux-143-qs-coverage-observability-and--t08-codex-mr9l88tm


🤖 Generated by Code UX

@numnx
numnx merged commit 8f65d40 into feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3 Jul 6, 2026
6 checks passed
@numnx
numnx deleted the task/feature-codux-143-qs-coverage-observability-and--t08-codex-mr9l88tm branch July 6, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant