Flake audit pass across all test suites

[Sootopolis]

What

Tracking issue: do a periodic audit of every `sbt test` suite for flake risks. Look for the usual culprits:

• Wall-clock dependencies (`Instant.now()` / `System.currentTimeMillis` in production code that tests exercise — use `Clock.instant`, run under `TestClock`).
• `provideShared` against a DB schema without `@@ TestAspect.sequential` when tests poll shared rows or share scheduler state.
• Real-clock `ZIO.sleep` / busy-waits without an explicit `@@ TestAspect.withLiveClock`.
• Cross-suite stdout/stderr or logger contention (e.g. `ProgressDisplay` writes from sibling suites).
• Tests that depend on iteration order of `Set` / `Map`.
• PK-collision-prone same-microsecond inserts.

Why

Several individual flakes already tracked:

• Flaky TestProgressBar under full-suite load #64 — TestProgressBar (single occurrence so far)
• `memory/project_flaky_job_scheduler_test.md` — parallel-DB-shared-state race
• `memory/project_flaky_recruitment_explore_test.md` — interrupted-recruit timing (resolved)
• `memory/project_flaky_client_timing_test.md` — wall-clock dependency (largely resolved)

A coordinated pass while these are fresh would catch the next class of flake before it bites in CI.

Out of scope

Not blocking on any specific issue. Pick up when there's a quiet window or a related refactor.

[Sootopolis]

New flake data (2026-06-12) + a determinism goal

Preference recorded: deterministic tests, not flaky-retry band-aids.

New observations this session

• TestCcasCompletion flaked once under full-suite load — its <shell> -n syntax checks spawn subprocesses, which transiently fail under fork pressure. Mitigated with @@ TestAspect.flaky on those three tests (in feat(cli): cached shell completion (bash/zsh/fish) #78). That's a band-aid to remove, not determinism.
• The .git/hooks/pre-push hook (sbt test) flaked once and blocked a push; the retry plus two manual sbt test runs were 791 green. So flakes now cost pushes, not just CI.

Root-cause classes (from the flake memories + these observations)

• Test / parallelExecution := true → cross-suite races: shared-DB state (provideShared without per-suite isolation), stdout/logger contention (ProgressDisplay, Flaky TestProgressBar under full-suite load #64), subprocess fork pressure (the completion -n checks).
• Wall-clock deps largely already removed via the TestClock migration.

Options to evaluate

1. Test / parallelExecution := false — simplest hard guarantee; cost is wall-clock (~25s → ~1–2min).
2. Surgical: per-suite DB isolation (unique schema/keys), assert bar output against a captured sink instead of real System.out, @@ sequential only the few subprocess/shared-state suites — keep parallelism elsewhere.
3. Remove the @@ flaky band-aids once (1) or (2) lands.

Goal: a green sbt test is deterministic, so neither CI nor the pre-push hook ever flakes. Related: #64.

[Sootopolis]

Resolved via #81: Test/parallelExecution := false.

Root cause was cross-suite contention on three process-global resources: OS process table / memory (TestCcasCompletion's <shell> -n subprocess fork pressure), JVM System.out (TestProgressBar's capture swap), and DB rows (already isolated per-suite via FreshSchemaLayer). @@ TestAspect.sequential only orders cases within a suite, so it can't fix cross-suite contention — parallelExecution is the only lever. Removed the @@ TestAspect.flaky band-aids from TestCcasCompletion.

Cost is small (suite is DB-I/O-bound): test phase ~17s → ~28s. Validated: 20 consecutive cold sbt test runs, 0 failures, + green pre-push hook.

Follow-up: the TestProgressBar global-stream dependency was then fully removed in #64 (sink injection, #82). Parallelism stays off — re-enabling would still need the TestCcasCompletion fork-pressure flake solved separately (sbt concurrency tag or dropping the subprocess spawn).