Skip to content

perf(ci): speed up PR feedback with transpile-only jest and parallel jobs#4374

Merged
TaprootFreak merged 4 commits into
developfrom
perf/ci-faster-pr-feedback
Jul 24, 2026
Merged

perf(ci): speed up PR feedback with transpile-only jest and parallel jobs#4374
TaprootFreak merged 4 commits into
developfrom
perf/ci-faster-pr-feedback

Conversation

@TaprootFreak

@TaprootFreak TaprootFreak commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

What & why

PR feedback was dominated by the Jest step (~646 s for ~298 suites in a representative run). This PR makes it substantially faster without removing or deferring a single test — which matters here, because profiling showed the slowness is fixed per-suite overhead spread across the whole suite (incl. the auth / money / accounting suites), not a few skippable tests.

Two levers, both keep every test and weaken no gate:

  1. ts-jest transpile-only (isolatedModules). ts-jest was re-running a full TypeScript type-check on every suite. That work is redundant with the type-check the pipeline already does — so it's removed from the transform. Measured −42 % of total test time.
  2. Parallel, sharded CI. api-pr.yaml is split into parallel jobs — checks (lint / format / build / type-check / audit), Bank Frick coverage, and a 3-way sharded test matrix — so PR feedback ≈ max(job) instead of one serial ~15-min run.

Root cause (measured, clean local run, Node 20, 300 suites / 5223 tests, all green)

  • ~79 % of total test time is fixed per-suite overhead (NestJS TestingModule bootstrap + ts-jest transform), not test logic (a 1-test controller suite still took ~39 s; the mean runtime of ≤3-test suites was 17.7 s).
  • Of that transform cost, the per-suite type-check was pure duplicated work.

Type-safety is preserved (important)

isolatedModules makes ts-jest stop type-checking. That would have created a real gap: tsconfig.build.json excludes **/*spec.ts, so nest build never type-checks test files — type errors in specs would go uncaught. So this PR adds a single project-wide type-check step (tsc --noEmit, base tsconfig, includes specs) to CI. Net effect: the whole project is type-checked once, instead of every spec being type-checked on every suite run. (Verified locally: npm run type-check is green, ~62 s.)

Measured before → after

This PR's own CI run (2-core ubuntu-latest), all jobs running in parallel:

Parallel job Time
Test (shard 1/3) 239 s
Test (shard 2/3) 247 s
Test (shard 3/3) 236 s
Build and checks (lint / format / build / type-check / audit) 190 s
Bank Frick coverage 131 s

Whole PR feedback = max(all jobs) ≈ 247 s (~4.1 min), down from ~15 min — about −73 %. The former single ~10.8-min Jest step is now three disjoint ~2.5-min shards (verified: 100 + 100 + 100 = 300 suites, zero overlap, every suite runs exactly once). The new type-check step is 30 s.

Isolating just the transform change (m5me, 16 workers): full-suite wall-clock 413 s → 241 s (−42 %), per-suite work sum 6772 s → 3927 s. Everything else here is parallelism, which converts that work reduction into wall-clock.

Frick coverage under transpile-only

isolatedModules emits the emitDecoratorMetadata helpers differently, which adds phantom uncovered branches on dependency-injected constructors and would red the strict 100 % Frick coverage gate. So the coverage run (small, dedicated, not perf-sensitive) compiles with full type info via tsconfig.coverage.json (isolatedModules: false); the gate stays exact at 100 %, while the main suite keeps the fast transpile-only path.

What this deliberately does NOT do

  • No test tag / no deferred suites. An earlier draft moved “tooling” suites behind a slow/full-ci tag; a review found several of them actually assert authentication (2FA-gated staff replies), authorization (ForbiddenException on foreign notes/templates) and PII-leak prevention (Telegram group-bind guard). Deferring those would have removed real protection from the PR lane, so the tag approach was dropped entirely. This change excludes or defers no spec; every suite still runs on every PR. (One unrelated, pre-existing describe.skip in an EVM-delegation spec is left untouched.)

CI / branch protection note

The single Build and test job is split into two parallel jobs, Build and checks and Test. develop/main currently define no required status checks, so this rename breaks nothing. If required checks are added later, require both Build and checks and Test.

Follow-ups (optional)

  • @swc/jest is an alternative transform engine (not a missing piece): it could shave the remaining per-suite transform cost with no extra runner, but needs a full-suite verification of NestJS decorator-metadata / DI first. Deliberately not swapped in — ts-jest transpile-only already captured the transform win safely.

Addresses the slow-PR-feedback problem from #4365 (approach differs deliberately: no tag, every test kept).

ts-jest re-ran a full type-check on every suite, which is redundant with the pipeline's type-check and was the dominant test cost. Enable transpile-only via tsconfig isolatedModules, add a project-wide type-check step so spec files stay type-checked (nest build excludes them), and split api-pr.yaml into parallel checks and test jobs. Every test still runs on every PR.
Transpile-only (isolatedModules) emits the emitDecoratorMetadata helpers differently, adding phantom uncovered branches on dependency-injected constructors that red the 100% Frick coverage gate. Compile the coverage run with full type info (tsconfig.coverage.json, isolatedModules: false) so the gate stays exact; the main suite keeps transpile-only.
Split the full suite into 3 disjoint jest shards and move the Frick coverage gate into its own parallel job, so PR feedback is ~max(shard, checks, coverage) instead of one ~9 min test job. Every suite still runs exactly once (shards are disjoint and cover all 300 suites).
@TaprootFreak TaprootFreak changed the title perf(ci): faster PR feedback via transpile-only jest and parallel jobs perf(ci): speed up PR feedback with transpile-only jest and parallel jobs Jul 24, 2026
Correct the coverageThreshold location (jest.frick.config.js, not package.json) and record why test:frick:cov compiles with full type info (tsconfig.coverage.json) while the main suite runs transpile-only.
@TaprootFreak

TaprootFreak commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Review summary

Reviewed in parallel across two lenses — logic/correctness and CONTRIBUTING conformance.

Review passes to zero findings: the correctness lens came back clean; the conformance lens surfaced three minor items — a non-imperative PR title, a stale package.jsonjest.frick.config.js reference in docs/bank-frick-operations.md §7, and a branch-prefix note. The first two were fixed and pushed; the branch prefix is left as-is (transient, deleted on merge). A final parallel two-lane re-review then confirmed zero findings on both lenses.

Verified:

  • Shards disjoint and complete: --shard=1/3|2/3|3/3 → 100 + 100 + 100 = 300 suites, no overlap — every spec runs exactly once, none dropped.
  • Frick 100% coverage gate kept exact: the coverage run compiles with full type info (tsconfig.coverage.json, isolatedModules: false) so transpile-only decorator-metadata emit doesn't skew branch coverage; the coverageThreshold block is unchanged.
  • No gate lost: every original CI step is retained; all jobs skip the develop → main release PR as before.

CI green across all parallel jobs (Build and checks · Bank Frick coverage · Test shard 1–3). Whole-PR feedback ~4 min (from ~15 min).

@TaprootFreak
TaprootFreak marked this pull request as ready for review July 24, 2026 18:19
@TaprootFreak
TaprootFreak merged commit 0d2b768 into develop Jul 24, 2026
11 checks passed
@TaprootFreak
TaprootFreak deleted the perf/ci-faster-pr-feedback branch July 24, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant