docs: document jest suite load-flakiness and worktree test gotchas - #372
Closed
RonenMars wants to merge 1 commit into
Closed
docs: document jest suite load-flakiness and worktree test gotchas#372RonenMars wants to merge 1 commit into
RonenMars wants to merge 1 commit into
Conversation
…kip-ci] [skip-ci]
RonenMars
added a commit
that referenced
this pull request
Aug 1, 2026
## Why The runbook described itself as a "live procedure — edit it as PRs land", but the chain it sequenced is gone. Verified 2026-08-01 — every PR it orchestrated is `CLOSED` with `mergedAt=null`, i.e. **closed, not merged**: ``` #339 #341 #343 #345 #346 #347 #353 #354 #355 #356 #357 #358 #359 #360 #361 #362 #363 #364 #368 #372 #373 #376 #385 #386 #387 ``` The work reached `main` through the integration branch instead, and the PRs were closed as redundant. The integration branch the runbook treated as proof the set could coexist, `integration-dev/v1.0.0-2026-07-22`, has been deleted. That invalidates the phase order, the per-PR loop, the pre-flight sweep, the decision to close `#355`, the conflict **A–I** resolutions, and the definition of done. The only PR row still accurate is `#291`, which is still open and still excluded by standing request. ## What changed The body becomes a tombstone: what the runbook was for, what actually happened, and a table pointing at where each still-true trap now lives. The file is kept rather than deleted so inbound links resolve and nobody rebuilds the same plan from the merge report. Two of its four traps were already in `docs/troubleshooting.md` (the `.claude/` worktree gotcha and serial `SessionScreen` verification). The other two are lifted there now under a new "CI signals" section, because both are repo-wide facts rather than properties of those branches: - a `[skip-ci]` head commit reports every required context green in seconds having run nothing - a stacked PR (base ≠ `main`) still gets full CI here, since `test.yml`'s `pull_request` trigger has no `branches:` filter Conflict **J** is obsolete for a separate reason: those `ios/Podfile.lock` checksums encode the checkout's absolute path, so resolving it was never meaningful. #469 added `scripts/reset-podfile-lock-path-noise.sh`, which drops that drift. ## Two live findings, one already fixed The runbook's `[skip-ci]` section claimed the required contexts were `Type check`, `Unit tests`, `Integration tests`, `Lint`. The ruleset actually required **only `Lint` and `Type check`** — unit and integration tests could not block a merge at all, which makes the `[skip-ci]` trap worse than documented, not better. Its one outstanding post-merge action (add `i18n` to required checks, never done since #368) is now complete: `Unit tests`, `Integration tests` and `i18n` were added to the `main` ruleset on 2026-08-01. Confirmed safe first — all three report `pass` on a real `chore/bump-**` version-bump PR (#472), so the deploy pipeline's bumps are not deadlocked. Note: `.github/workflows/test.yml:25` carries the same wrong claim about which contexts are required. Left alone here to keep this docs-only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two testing gotchas cost real time during the 2026-07-22 integration snapshot and were only recorded in a throwaway snapshot branch.
This moves them somewhere durable.
Batch failures that aren't real — and the ones that are
The
SessionScreen.*suites each render the full session screen with fake timers, WebSocket stubs and long backstop timeouts.Jest's default parallel workers oversubscribe some machines, so a batch run reports failures that a single-suite run does not.
The trap runs in both directions, which is the part worth writing down.
During the snapshot, four genuinely broken suites were nearly dismissed as flakes on this basis, while two others really were load artifacts — and the batch output looked the same either way.
The rule is therefore not "use
--runInBand" but "never classify a batch failure without re-running that suite alone": passes alone means artifact, fails alone means real.Also lists the suites currently known to be load-sensitive, so they are not re-investigated from scratch.
npx jestfinds 0 tests in a worktree under.claude/testPathIgnorePatternsexcludes\.claude\, so every test path inside such a worktree is filtered out and scripts likenpm run test:i18nlook broken when they are not.Documents the fix (create worktrees outside
.claude/), the override for running in place, and the caveat that the override re-enables__tests__/unit/scripts/, which the main config excludes deliberately.Scope
Documentation only — no code, config or workflow changes.
docs/troubleshooting.mdgains a "Jest test suites" section matching the existing symptom / When / Cause / Fix format, andCLAUDE.mdgains a short rule pointing at it.