Skip to content

Codify Deterministic Test Fixtures (codex) - #2377

Merged
numnx merged 1 commit into
feature/CODUX-144-qs-coverage-observability-and-ci-reliability-audit-4from
task/feature-codux-144-qs-coverage-observability-and--t02-codex-mr9nlx4a
Jul 6, 2026
Merged

Codify Deterministic Test Fixtures (codex)#2377
numnx merged 1 commit into
feature/CODUX-144-qs-coverage-observability-and-ci-reliability-audit-4from
task/feature-codux-144-qs-coverage-observability-and--t02-codex-mr9nlx4a

Conversation

@numnx

@numnx numnx commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

🧩 Task: Codify Deterministic Test Fixtures

Automated task PR opened by Code UX for T02.

📋 Summary

Sprint: Sprint 144 · QS: Coverage Observability and CI Reliability Audit · Task: T02

Create a repeatable coverage, observability, security, and CI reliability quicksprint by adding enforceable tests, CI/Playwright hardening, structured logging guarantees, and aligned operational documentation without lowering coverage thresholds.

📝 Full Task Prompt
## Objective
Close the current verification gap where deterministic test assumptions live partly in convention and can regress silently. The desired guarantee is that Vitest setup keeps isolated app state, UTC/C locale defaults, explicit fake-timer use, and safe cleanup behavior across backend tests.

## Scope
- Edit `tests/setup/runtime-warning-filter.ts` only if a helper or exported assertion metadata is needed
- Create or edit `tests/backend/scripts/test-determinism-guardrails.test.ts`
- Verify `vitest.config.ts`
- Verify representative tests under `tests/backend/shared/polling/wait-until.test.ts`, `tests/backend/repositories/sqlite-connection.test.ts`, and `tests/backend/services/provider-concurrency-service.test.ts`
- Relevant symbols: `withIsolatedTestHome`, `VITEST_IN_MEMORY_DB`, `HOME`, `USERPROFILE`, `XDG_CONFIG_HOME`, `XDG_STATE_HOME`, `XDG_CACHE_HOME`, `vi.useFakeTimers`, `vi.useRealTimers`

## Implementation Requirements
1. Add tests that exercise `withIsolatedTestHome` and assert it sets and restores `HOME`, `USERPROFILE`, `XDG_CONFIG_HOME`, `XDG_STATE_HOME`, and `XDG_CACHE_HOME` for both successful and throwing callbacks.
2. Add a repository-level guard test that reads `vitest.config.ts` and asserts deterministic locale/timezone settings are present and that `environment` remains `node` by default.
3. Add a guard test that scans backend test files for `vi.useFakeTimers()` and requires nearby `vi.useRealTimers()` or `afterEach` cleanup in the same file, with an allowlist only if a file has a documented reason.
4. Add a guard test that verifies tests using filesystem app state or SQLite helpers either rely on `VITEST_IN_MEMORY_DB=true` or use `withIsolatedTestHome`/temporary directories instead of a real user home.
5. Keep failure output actionable by listing the exact offending file paths and the missing cleanup category.

## Constraints
- Do not add brittle line-number assertions.
- Do not require fake timers globally; the intended guarantee is explicit fake-timer setup and cleanup per test file.
- Do not invoke real Docker, Git remotes, provider CLIs, or persistent SQLite files.
- Preserve current runtime warning filtering behavior for ordinary test runs.

## Verification
- Run `pnpm run test:backend -- tests/backend/scripts/test-determinism-guardrails.test.ts` or the exact focused file created.
- Run `pnpm run test:backend -- tests/backend/shared/polling/wait-until.test.ts tests/backend/repositories/sqlite-connection.test.ts tests/backend/services/provider-concurrency-service.test.ts`.
- Run `pnpm run lint` and confirm the guardrail tests have meaningful failure messages.

✅ QA Review

⏳ QA review runs after this PR is opened. Check the task card in the dashboard for the outcome once it lands.

🤖 Model & Provider

Provider Codex CLI
Model gpt-5.5

⏱️ Timing

Started 2026-07-06 20:09:41 UTC
Finished 2026-07-06 20:15:34 UTC
Duration 5m 53s

📊 Token Usage

Metric Value
Input tokens 112,497
Cached input tokens 1,286,784
Output tokens 10,156
Total tokens 1,409,437
Invocations 1
Tool calls 41
Active time 5m 14s

Included usage estimate (subscription/local login): $1.51

🌿 Branch Info

Base: feature/CODUX-144-qs-coverage-observability-and-ci-reliability-audit-4
Head: task/feature-codux-144-qs-coverage-observability-and--t02-codex-mr9nlx4a


🤖 Generated by Code UX

@numnx
numnx merged commit bbb48a7 into feature/CODUX-144-qs-coverage-observability-and-ci-reliability-audit-4 Jul 6, 2026
6 checks passed
@numnx
numnx deleted the task/feature-codux-144-qs-coverage-observability-and--t02-codex-mr9nlx4a branch July 6, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant