Skip to content

Add CI workflow health assertions (codex) - #2370

Merged
numnx merged 2 commits into
feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3from
task/feature-codux-143-qs-coverage-observability-and--t09-codex-mr9l88xn
Jul 6, 2026
Merged

Add CI workflow health assertions (codex)#2370
numnx merged 2 commits into
feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3from
task/feature-codux-143-qs-coverage-observability-and--t09-codex-mr9l88xn

Conversation

@numnx

@numnx numnx commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

🧩 Task: Add CI workflow health assertions

Automated task PR opened by Code UX for T09.

📋 Summary

Sprint: Sprint 143 · QS: Coverage Observability and CI Reliability Audit · Task: T09

Strengthen Code UX coverage observability and CI reliability through enforceable tests, workflow hardening, structured observability guarantees, security validation, and aligned operational documentation without lowering coverage thresholds.

📝 Full Task Prompt
## Objective
Make CI workflow reliability auditable by testing key workflow invariants for package manager version, Node version, cache keys, install mode, audit job, concurrency cancellation, and release lane separation.

## Scope
- `.github/workflows/ci.yml`
- `.github/workflows/playwright.yml`
- `.github/workflows/release-checks.yml` if present
- `package.json`
- `tests/backend/ci/workflow-health.test.ts` or `tests/backend/docs/testing-docs-commands.test.ts`
- `docs/development/testing-and-quality.md`
- `docs/operations/runbook.md`
- Current verification gap: workflow YAML is manually reviewed, but the audit needs automated assertions that prevent accidental drift in CI reliability settings.
- Desired guarantee: CI continues to use pnpm 10.33.0, Node 22, frozen lockfile installs, stable cache keys, independent audit, concurrency cancellation, and separate release/Playwright validation lanes.

## Implementation Requirements
1. Add a backend test that reads workflow YAML as text or a structured object using existing dependencies and asserts `pnpm/action-setup` uses version `10.33.0`, `actions/setup-node` uses Node `22`, and install steps run `pnpm install --frozen-lockfile --ignore-scripts`.
2. Assert `.github/workflows/ci.yml` has `concurrency.cancel-in-progress: true`, separate jobs for typecheck, backend coverage, dashboard tests, build, and security audit, and `pnpm run audit` runs independently.
3. Assert cache keys include OS, Node 22, pnpm 10.33.0, and dependency/config hash inputs relevant to each job.
4. Assert `.github/workflows/playwright.yml` builds before running `pnpm run test:e2e`, installs Chromium after cache restore, uploads `test-results/` and `playwright-report/` with `if: always()`, and remains scoped to release-path branches unless intentionally changed.
5. Update `docs/development/testing-and-quality.md` and `docs/operations/runbook.md` to match the tested CI invariants and local equivalent commands.

## Constraints
- Do not add new YAML parser dependencies; use existing runtime capabilities or text assertions if no parser exists.
- Do not merge Playwright, release, audit, and core CI lanes into one job.
- Preserve frozen lockfile installs and `--ignore-scripts` unless repository policy changes elsewhere.
- Keep assertions resilient to job ordering but strict about required invariants.

## Verification
- `pnpm run test:backend -- tests/backend/ci/workflow-health.test.ts tests/backend/docs/testing-docs-commands.test.ts`
- `pnpm run lint`
- `pnpm run build`
- `pnpm run ci`
- Success means workflow drift breaks a focused test before CI reliability settings are weakened.

✅ QA Review

⏳ QA review runs after this PR is opened. Check the task card in the dashboard for the outcome once it lands.

🤖 Model & Provider

Provider Codex CLI
Model gpt-5.5

⏱️ Timing

Started 2026-07-06 19:03:05 UTC
Finished 2026-07-06 19:15:19 UTC
Duration 12m 14s

📊 Token Usage

Metric Value
Input tokens 120,862
Cached input tokens 2,252,544
Output tokens 10,061
Total tokens 2,383,467
Invocations 1
Tool calls 48
Active time 10m 13s

Included usage estimate (subscription/local login): $2.03

🌿 Branch Info

Base: feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3
Head: task/feature-codux-143-qs-coverage-observability-and--t09-codex-mr9l88xn


🤖 Generated by Code UX

Code UX added 2 commits July 6, 2026 19:15
…i-reliability-audit-3 into task/feature-codux-143-qs-coverage-observability-and--t09-codex-mr9l88xn
@numnx
numnx merged commit f83a943 into feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3 Jul 6, 2026
6 checks passed
@numnx
numnx deleted the task/feature-codux-143-qs-coverage-observability-and--t09-codex-mr9l88xn branch July 6, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant