test!: mock-free test architecture - #359
Merged
Merged
Conversation
Workstream 1 of the production-hardening effort. Establishes the rule "assert effects, never call shapes", the three-tier taxonomy, and the bundle-cache fixture architecture that makes migrating 77 tests to the real time-skipping server affordable. Grounded in measured state: 12 spec files mock the Temporal SDK across 241 tests, while worker unit coverage reports 93.7% statements — largely against a hand-written fake. PR #357's isThenable bug is the precedent for why that number is not trustworthy. Enforcement is a meta-test rather than a lint rule: the repo lints with oxlint only, which cannot express the check, and adding ESLint for a single rule is disproportionate.
11 tasks: bundle-cache helpers, the no-sdk-mocks guard, six spec migrations, the worker.spec split, nightly Stryker, and final verification. Records two API facts discovered while planning that the spec had wrong or omitted: CreateWorkerOptions omits `taskQueue` and TypedWorker.create hard-codes `contract.taskQueue`, so per-test isolation must scope the contract rather than the worker options; and registration verification only runs for `workflowsPath`, so the registration specs must not use the bundle cache.
- Add tests asserting bundleFor returns the same promise instance for repeated calls on one path (proving single invocation without mocking the Temporal SDK) and different instances for different paths. - Correct workflow-bundle.ts JSDoc: the cache lives for one test file under Vitest's default isolation, not for the Vitest worker process.
The guard scanned only packages/, leaving examples/ and docs/ unmonitored despite the design doc specifying a workspace-wide walk. Skip dot-directories (in addition to node_modules and dist) so the broadened walk doesn't descend into docs/.vitepress/dist or similar build caches.
…point Dynamically loaded via bundleFor(workflowPath(...)), like its sibling *.workflows.ts fixtures — knip's static analysis can't see that reference, so it needs the same entry-list treatment.
…strengthen timeout assertion
Addresses code review on the activity-options migration:
- packages/worker/src/internal.spec.ts covers the two ContractMisuseError
throws in buildRawActivitiesProxy with real ActivityOptions and no SDK
mock — both are reachable before/without ever invoking a proxied
activity function, so no Temporal environment is needed.
- activity-options.contract.ts/.workflows.ts add a second contract
(layeredOptionsContract) and workflow (resolvesLayeredOptions) proving,
as real effects on the time-skipping server, the five merge/precedence
behaviors the deleted workflow-proxy.spec.ts only asserted as call
shapes: activityOptionsByName precedence over contract-level options,
contract-level options over the workflow-wide default, per-activity
routing between customized and default proxies, shallow (not deep)
merging of the retry block across layers, and the same merge applying
to global (contract-level) activities.
- The original timeout test's assertion is strengthened from
`toContain("err:")` (true for any Err — validation failure, missing
activity, ...) to the exact `toBe("err:START_TO_CLOSE")`, narrowing the
activity error's unwrapped cause to a real TimeoutFailure.
Every new/changed assertion was verified to fail for the right reason by
temporarily breaking the corresponding behavior and restoring it.
Replaces the mocked-SDK handlers.spec.ts (28 tests against a hand-rolled setHandler/defineSignal/defineQuery/defineUpdate stub) with an in-process suite (6 tests) that starts a real workflow on the time-skipping test server and drives it through TypedWorker/TypedClient. Covers the 7 behaviors that must survive: valid signal delivery, invalid signal drop-and-log, valid query output, worker-side query input validation, valid update execution, update pre-admission rejection with history left unaffected, and bind-time ContractMisuseError for an async-validating query schema. The update/query/signal drop tests route through `handle.raw` (the real @temporalio/client WorkflowHandle) rather than the typed client, because the typed client validates client-side with the identical schema and would otherwise reject bad input before it ever reached the worker -- defeating the point of proving the worker's own validation path. The update-rejection test inspects real workflow history (fetchHistory) to prove no update-admission event was written, which is the one guarantee the mocked setHandler could never make.
….spec.ts migration Code review on the prior commit found 8 tests dropped with no replacement (query/update output validation, the async-output-schema query/update asymmetry, and the three schema-probe edge cases — including the non-Promise-thenable guard, which is load-bearing: that exact bug shipped before and was caught late), plus 3 wire-format "receiving side" tests wrongly deferred to a later task that never covered them. Also fixes a vacuously-passable history assertion (no positive control) and a signal-drop regression that would hang instead of failing fast. - handlers.contract.ts: adds a payload-less `finish` signal (replaces the `total >= 10` termination so a `bump` regression fails fast instead of hanging), `brokenOutput`/`brokenOutputUpdate` (deliberately invalid handler returns), `asyncOutputUpdate` (the query/update output-schema asymmetry), `bindsAsyncQueryOutputSchema` and `probeEdgeCases` workflows for the three probe edge cases, and `transformWorkflow` for the wire-format signal/query/update tests. - handlers.inprocess.spec.ts: 14 tests total. Captures workflow `log.warn` output via `Runtime.logger` (the SDK's documented hook, not the reserved `__temporal_logger` sink) to assert the drop is logged, not just non-fatal. Adds a positive control (WorkflowExecutionStarted) to the update pre-admission history assertion so a future `eventType` decoding change fails loudly instead of passing vacuously.
bindUpdateHandler runs its own assertSyncSchema(updateDef.input, ...) call, a separate call site from bindQueryHandler's. The previous commit only restored coverage for the query-input and query-output variants; this adds the update-input one (bindsAsyncUpdateSchema workflow) so a regression at that specific call site is no longer unaccounted for in the disposition table.
Replace the mocked cancellation.spec.ts (13 tests, vi.mock("@temporalio/workflow"))
with real time-skipping-server coverage: the swallowed-cancellation hazard (a
declared-errors activity turns a cancel into an absorbable Err), the
rethrowCancellation fix, a nonCancellable scope shielding an outer cancel, and
cancellableScope/nonCancellableScope's own Result-folding mechanics (success,
defect classification, an internally-raised real CancelledFailure). Adds
@temporalio/activity as a devDependency to build a heartbeating, genuinely
cancellable test activity.
Restore cause-preservation assertions dropped in the prior migration (exact folded messages for defect/manufactured-cancellation modes, a structural CancelledFailure check for the real-server cancellation path), cap slowActivity's retries to match the sibling contract, remove every throw-on-defect hang risk in the cancellation test workflows in favor of a terminal, assertable status, and bound every started workflow with workflowExecutionTimeout so a context-wiring regression fails in ~30s instead of hanging to the 120s test timeout.
Migrates continue-as-new.spec.ts (mocked makeContinueAsNewFunc) to a real
time-skipping server. The mock could only prove a stub was called with a
particular options shape; it could never prove the next run actually
receives the carried state, or that a smuggled workflowType/taskQueue
override in the options bag is powerless against the validated target.
Adds continue-as-new.{contract,workflows,inprocess.spec}.ts exercising:
same-workflow state carry-over across two continue-as-new boundaries,
options (memo) forwarding, same-workflow input validation, the D1
wire-format guarantee (original args, not the schema-parsed value, cross
the wire), cross-contract dispatch (valid/invalid-args/undeclared-target),
and the dispatch heuristic's two misrouting traps. Every workflow start
carries a 30s workflowExecutionTimeout so a regression fails a fast,
bounded assertion instead of hanging to the suite's 120s timeout.
Deletes the mocked spec and its no-sdk-mocks.spec.ts allowlist entry.
… gap Review found that createContinueAsNew's pre-send input validation (internal.ts:298-302) could be deleted with the suite still green: the two "rejects invalid args" tests only asserted failure type/message, both of which are reproduced character-for-character by the RECEIVING run's own incoming-input validation once the invalid args cross the wire. Add a run-identity assertion (handle.raw.describe().runId === handle.firstExecutionRunId) to both tests, proving no continuation ever reached Temporal — deleting the guard now fails on that assertion specifically, confirmed by reality check. Also: dispatchHeuristic now returns a hop count so its two tests assert exactly one continuation happened, not just "completed"; consolidated the shared-static-task-queue rationale into one doc comment instead of duplicating it (and inconsistently omitting it) at each call site; and corrected a comment overclaiming "almost instant" fast-forwarding against the ~31s the reality checks actually measured.
Migrates wire-format.spec.ts (8 tests) off the mocked Temporal SDK. Replaces call-shape assertions on faked executeChild/startChild with real time-skipping-server executions proving the v8 single-parse guarantee at the workflow entry point and the child-workflow boundary (executeChildWorkflow/startChildWorkflow/child signals).
Review finding: the child-wire migration restored firstExecutionRunId
coverage but left workflowId unasserted — child-workflow.ts's
`workflowId: handle.workflowId` passthrough could return any string
with every test still green, since the replacement only echoed back
the parent's own locally-computed id. parentChild/parentSignal now
return handle.workflowId itself, and the spec asserts it directly plus
uses it (instead of a duplicated derivation) to independently look up
the child's real execution.
Also drops a redundant not.toBe("") assertion sitting next to a
stronger toBe(actualRunId) check.
Replaces workflow-errors.spec.ts's mocked proxyActivities/workflowInfo with real end-to-end coverage in rehydration.inprocess.spec.ts: a message-forwarding assertion on the existing skew-control case, plus two new tests proving declareWorkflow's catch block fails fast on invalid contract-error data (ContractErrorDataValidationError) and rethrows non-ContractError throws untouched. Both existing tests switch from per-test workflowsPath bundling to bundleFor.
…ed cause
Review found the previous commit's disposition table silent on `details`
(the {$tc:1} wire marker) and its `nonRetryable` cost claim wrong. Both
are now asserted directly off ContractError.cause (the raw
ApplicationFailure rehydration preserves there), closing a real gap:
_internal_rehydrateContractError only gates on the wire marker for
data-less errors, so a data-carrying error's marker emission had no
coverage. Also fixes the file header's scenario numbering to stop
implying items 3-4 continue the 2-item rehydration-gap list they don't
belong to.
worker.spec.ts's 18 mocked tests split by governing rule: pure option-mapping stays in unit (3 workflowsPathFromURL tests, no SDK mock needed), Temporal configuration/behavior moves to inprocess. Registration-check coverage moves to a new registration.inprocess.spec.ts using real workflowsPath fixtures (never bundleFor, which skips the check by design). Adds real-effect coverage for arbitrary WorkerOptions passthrough (identity, read back from Temporal's own history) and run()'s failure path (a genuine double-run IllegalStateError — Worker.create's real API is async, so this single real trigger subsumes the mocked suite's separate sync-throw/async-reject scenarios). Deletes worker.spec.ts's no-sdk-mocks allowlist entry — the last one in the file.
…ering Review of the Task 9 split found row 2's bundling-failure test dropped the original mock's cause.cause assertion with no successor — a reality-check break (deleting the cause argument in worker.ts's create() qualifier) passed every test in the repo. Restores it as an instance-type + message-substring check (identity isn't reproducible for a real bundler error). Also moves the double-run test's shutdown/cleanup into a finally block so a failing assertion — the exact regression the test exists to catch — can't leak a live worker in the process-scoped time-skipping environment.
Adds Stryker (core + vitest-runner) scoped to the pure, non-sandboxed modules that own the most error-prone logic: contract/builder.ts, contract/errors-impl.ts, worker/contract-errors.ts, worker/error-tags.ts, worker/internal.ts. Sandboxed workflow paths are excluded — their run cost under mutation testing is prohibitive. The installed @stryker-mutator/vitest-runner@9.2.0 has no `vitest.project` option (only `configFile`/`dir`/`related`), so a dedicated vitest.stryker.config.ts composes just the "unit" projects of the contract and worker packages, keeping the heavier integration/sandboxed-workflow suites out of the mutation run entirely. "break": null is deliberate: this establishes a baseline, not a gate. Observed baseline (pnpm test:mutation): 61.45% overall (483/786 killed), with worker/error-tags.ts (28.57%) and worker/internal.ts (33.53%) the weakest spots — see task-10-report.md for surviving-mutant detail. Runs nightly via .github/workflows/mutation.yml (cron + workflow_dispatch), not per-PR.
…he report test:mutation ran stryker directly, bypassing turbo's build dependency graph. Worker's unit specs import @temporal-contract/contract by package name, which resolves through its exports map to dist/ — gitignored and built by nothing else in the nightly workflow's setup step. On a fresh runner the Stryker dry run would have failed before producing a score. - package.json: test:mutation is now "pnpm build && stryker run". - .github/workflows/mutation.yml: add timeout-minutes: 30 (an unbounded nightly job risks burning GitHub's 6-hour default on a hang), and upload reports/mutation/ as a build artifact (the html reporter's output was otherwise generated and discarded every run). Verified cold: with packages/*/dist removed, `pnpm test:mutation` rebuilds everything and reproduces the 61.45% baseline (see task-10-report.md).
…ting routing.spec.ts (Docker tier) timed out reproducibly against the shared 10s testTimeout because a single Vitest invocation scheduled it alongside 9 inprocess files, each spawning its own TestWorkflowEnvironment test-server process. Split test:integration into two sequential Vitest invocations so each tier gets its own parallelism budget instead of competing with the other's process-spawning cost in the same run.
bindSignalHandler/bindQueryHandler/bindUpdateHandler each guard a missing declaration block and an unknown name, throwing a non-retryable ApplicationFailure a regression could silently turn into a plain Error (infinite Workflow Task retry). Previously only asserted indirectly via a hand-constructed message string in errors.spec.ts, never through the production guards themselves. Modeled on internal.spec.ts: every throw fires before setHandler binds a real handler, so none of the six tests need a Temporal environment or an SDK mock. Each asserts the exact message via toBe, not just the error class, so a regression that swapped two of the six messages would still be caught.
…s walk
SDK_MOCK missed vi.doMock("@temporalio/...") and the type-safe
vi.mock(import("@temporalio/...")) form Vitest's docs recommend. Widen
the regex to catch both, verified against throwaway specs for each form.
Also assert the directory walk actually found more than 20 spec files,
so a future change to the skip logic in specFiles can't silently narrow
it to nothing and have the "no offenders" assertion pass vacuously.
…c site docs/superpowers/plans and docs/superpowers/specs are working documents for this repo's own development process. Without srcExclude, VitePress would build and deploy-docs.yml would publish them alongside real docs on merge to main.
The subpath export shipping bundleFor/withTaskQueue/nextTaskQueueId had neither. Add the missing changeset (minor: purely additive) and add src/workflow-bundle.ts to typedoc.json's entryPoints so it's no longer the only exported subpath absent from generated API docs.
internal.ts's 33.53% baseline is partly a scope artifact: the mock-free suites under packages/worker/src/__tests__/ (including rehydration.inprocess.spec.ts, which exercises the ChildWorkflowFailure branch) are excluded from vitest.stryker.config.ts's composed "unit" projects. error-tags.ts's low score, by contrast, is a genuine gap. This caveat previously existed only in the gitignored SDD ledger; move it into mutation.yml so it survives independent of that scratch space.
…flow-bundle workflowPath (and, in one file, fixturePath) was copy-pasted verbatim into 12 spec files across packages/worker and packages/client, always resolving a sibling fixture relative to the caller's own import.meta.url. @temporal-contract/testing/workflow-bundle already exists and is already imported by most of those files, making it the natural shared home. The lifted fixturePath(callerUrl, filename) takes the caller's import.meta.url as an explicit parameter rather than reading its own — the whole point is resolving relative to where the *caller* lives, and that only works if the caller supplies its own URL. Added direct unit coverage in workflow-bundle.spec.ts (previously untested despite 12 copies) and updated all 12 call sites, including five that didn't already import from workflow-bundle at all (they call Worker.create/TypedWorker.create with workflowsPath directly).
The activityOptionsByName override in activity-options.workflows.ts shallow-replaces flakyActivity's contract-level retry block (by design, to prove shallow merge doesn't inherit maximumAttempts from the layer below) but supplied no maximumAttempts of its own, leaving the EFFECTIVE policy Temporal actually runs with unbounded. A regression that broke the "succeeds on retry" path would hang the test to its 120s timeout instead of failing an assertion. Add an explicit maximumAttempts: 2 to the override — exactly what the "fails once, succeeds on retry" implementation needs — without defeating the shallow-merge proof: the override's own bound (2) still visibly wins over the contract-level one (1), it's just no longer implicitly unbounded in between. Also soften activity-options.contract.ts's doc comment, which asserted from an informal, unreproduced probe that the time-skipping server skips retry backoff delay. That claim doesn't matter to correctness anymore: the explicit cap bounds retries regardless of backoff timing.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements Workstream 1 (“mock-free test architecture”) by migrating Temporal-SDK-mocking specs to effect-based tests that run against a real time-skipping Temporal test server, adding enforcement to prevent reintroducing SDK semantic mocks, and introducing nightly mutation testing (Stryker) as a baseline metric.
Changes:
- Replaces several worker specs that mocked
@temporalio/*with*.inprocess.spec.tssuites that assert real Temporal effects (timeouts, cancellation semantics, continue-as-new state, wire-format single-parse behavior, registration verification). - Adds
@temporal-contract/testing/workflow-bundleutilities to memoize workflow bundling per test file and centralize fixture path resolution + task-queue scoping. - Adds Stryker configuration + a nightly GitHub Actions workflow to produce a mutation-testing baseline (not a gating threshold).
Reviewed changes
Copilot reviewed 49 out of 51 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| vitest.stryker.config.ts | Dedicated Vitest config for Stryker runs, composing only unit projects. |
| stryker.config.json | Stryker runner configuration, mutation scope, and thresholds baseline setup. |
| pnpm-workspace.yaml | Adds catalog entries for Stryker packages and @temporalio/activity. |
| packages/worker/src/workflow-proxy.spec.ts | Removes SDK-mocked call-shape unit spec (replaced by effect-based tests elsewhere). |
| packages/worker/src/workflow-errors.spec.ts | Removes SDK-mocked workflow error conversion unit spec (coverage moved to in-process rehydration suite). |
| packages/worker/src/worker.spec.ts | Keeps only pure workflowsPathFromURL tests; moves behavioral worker coverage to in-process specs. |
| packages/worker/src/wire-format.spec.ts | Removes SDK-mocked wire-format spec (replaced by in-process child-wire suite). |
| packages/worker/src/internal.spec.ts | Adds non-mocked runtime coverage for buildRawActivitiesProxy misuse guard paths. |
| packages/worker/src/continue-as-new.spec.ts | Removes SDK-mocked continue-as-new unit spec (replaced by in-process continue-as-new suite). |
| packages/worker/src/cancellation.spec.ts | Removes SDK-mocked cancellation unit spec (replaced by in-process cancellation suite). |
| packages/worker/src/tests/worker.spec.ts | Switches workflow fixture path helper to shared fixturePath. |
| packages/worker/src/tests/time-skipping.inprocess.spec.ts | Uses shared fixturePath and extends real-server assertions (incl. history-based checks). |
| packages/worker/src/tests/routing.spec.ts | Uses shared fixturePath and removes local path helper duplication. |
| packages/worker/src/tests/replay.inprocess.spec.ts | Uses shared fixturePath and removes local path helper duplication. |
| packages/worker/src/tests/rehydration.workflows.ts | Expands workflow modes to cover additional rehydration/error-conversion behaviors on real server. |
| packages/worker/src/tests/rehydration.inprocess.spec.ts | Switches to bundle caching + per-test task queues; adds contract-error conversion coverage. |
| packages/worker/src/tests/rehydration.contract.ts | Adds declared error message for end-to-end message preservation assertions. |
| packages/worker/src/tests/registration.inprocess.spec.ts | New real-server coverage for workflow registration completeness check behavior. |
| packages/worker/src/tests/registration.contract.ts | Updates comment to reflect in-process registration spec location/purpose. |
| packages/worker/src/tests/handlers.workflows.ts | New workflow fixtures to drive handler behavior (signals/queries/updates) against real server. |
| packages/worker/src/tests/handlers.contract.ts | New contract defining handler surfaces and schema edge cases for in-process tests. |
| packages/worker/src/tests/continue-as-new.workflows.ts | New workflow fixtures to drive continue-as-new semantics against real server. |
| packages/worker/src/tests/continue-as-new.inprocess.spec.ts | New in-process suite asserting real continue-as-new effects and guard behaviors. |
| packages/worker/src/tests/continue-as-new.contract.ts | New contracts used by continue-as-new in-process suite. |
| packages/worker/src/tests/child-wire.workflows.ts | New workflow fixtures to drive entry/child wire-format effects against real server. |
| packages/worker/src/tests/child-wire.inprocess.spec.ts | New in-process suite for entry + child workflow wire-format single-parse guarantees. |
| packages/worker/src/tests/child-wire.contract.ts | New contract defining transforming schemas and child boundary scenarios. |
| packages/worker/src/tests/cancellation.workflows.ts | New workflow fixtures to demonstrate swallowed-cancellation hazard + fix on real server. |
| packages/worker/src/tests/cancellation.inprocess.spec.ts | New in-process suite asserting real cancellation propagation and scope behaviors. |
| packages/worker/src/tests/cancellation.contract.ts | New contract defining cancellation workflows + activity options used in cancellation suite. |
| packages/worker/src/tests/activity-options.workflows.ts | New workflow fixtures to make activity-option effects observable (e.g., real timeouts). |
| packages/worker/src/tests/activity-options.inprocess.spec.ts | New in-process suite proving activityOptions merge/precedence via real server behavior. |
| packages/worker/src/tests/activity-options.contract.ts | New contracts for activity-options effect assertions and merge-precedence scenarios. |
| packages/worker/package.json | Runs integration tiers sequentially; adds @temporalio/activity dev dependency. |
| packages/testing/typedoc.json | Adds workflow-bundle.ts to typedoc entry points. |
| packages/testing/src/workflow-bundle.ts | Introduces bundle memoization + task-queue scoping + fixture path helper utilities. |
| packages/testing/src/workflow-bundle.spec.ts | Unit tests for workflow-bundle helper utilities. |
| packages/testing/src/no-sdk-mocks.spec.ts | Adds a meta-test enforcing “no Temporal SDK mocks” outside a documented allowlist. |
| packages/testing/package.json | Exports/builds new ./workflow-bundle subpath. |
| packages/client/src/tests/client.spec.ts | Switches to shared fixturePath helper. |
| package.json | Adds test:mutation script; ensures vitest is catalog-sourced at root. |
| knip.json | Adds vitest.stryker.config.ts as an entry; simplifies worker workflows fixture entries glob. |
| docs/superpowers/specs/2026-08-01-mock-free-test-architecture-design.md | Adds the approved design spec documenting goals, tiers, and enforcement. |
| docs/.vitepress/config.ts | Excludes internal docs/superpowers/** content from published docs output. |
| .gitignore | Ignores Stryker temp/log/report outputs. |
| .github/workflows/mutation.yml | Adds nightly mutation testing workflow that uploads the HTML report artifact. |
| .changeset/testing-workflow-bundle-export.md | Changeset documenting the new testing subpath export. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
…losed one The Security Audit job failed on PR #359 with 7 findings. Attribution: - minimatch 10.x (3x High, ReDoS) via @stryker-mutator/core - ajv 8.x (Moderate, ReDoS via `$data`) via @stryker-mutator/core - @babel/core (Low, arbitrary file read) via @stryker-mutator/core Those three arrived with the nightly mutation runner added in this branch, so they are ours to fix. Each override is version-scoped to the line already present rather than lifted across a major, matching the existing entries. The advisory for @babel/core names 7.29.1 as the first patched release, but that version was never published — the 7.29.x line goes 7.29.0 -> 7.29.6 -> 7.29.7 — so the override pins the head of the 7.x line instead. Also patches dompurify (Low) via docs > mermaid, which is NOT from this branch: the advisory was disclosed after main's last green audit, so main is affected too. Included because it blocks this PR's audit job and the fix is a version-scoped override in the same style; `build:docs` is verified against it. `pnpm audit` now exits 0, with only the pre-existing GHSA-mh99-v99m-4gvg ignore remaining. typecheck/lint/test 16/16, build:docs 9/9.
`relative()` returns host-specific separators, so on Windows the offender
check produced `packages\testing\…` and matched none of the forward-slash
ALLOWLIST keys. Every allowlisted spec would have been reported as an
offender and the guard would have failed for the wrong reason.
That direction is the safe one — a false failure, not a false pass — but a
guard that cries wolf is a guard someone disables, and this one is now the
only thing preventing SDK mocks from returning.
Normalizes to POSIX form before the allowlist lookup. The offender list is
reported in the same form, so its output stays copy-pasteable into the
allowlist on any platform.
The sibling stale-entry check needed no change: it feeds allowlist keys
through `join()`, which normalizes forward slashes on Windows already.
Verified the guard still fails on a planted `vi.mock("@temporalio/workflow")`
and names the file in POSIX form.
Reported by Copilot on #359.
btravers
added a commit
that referenced
this pull request
Aug 2, 2026
…losed one The Security Audit job failed on PR #359 with 7 findings. Attribution: - minimatch 10.x (3x High, ReDoS) via @stryker-mutator/core - ajv 8.x (Moderate, ReDoS via `$data`) via @stryker-mutator/core - @babel/core (Low, arbitrary file read) via @stryker-mutator/core Those three arrived with the nightly mutation runner added in this branch, so they are ours to fix. Each override is version-scoped to the line already present rather than lifted across a major, matching the existing entries. The advisory for @babel/core names 7.29.1 as the first patched release, but that version was never published — the 7.29.x line goes 7.29.0 -> 7.29.6 -> 7.29.7 — so the override pins the head of the 7.x line instead. Also patches dompurify (Low) via docs > mermaid, which is NOT from this branch: the advisory was disclosed after main's last green audit, so main is affected too. Included because it blocks this PR's audit job and the fix is a version-scoped override in the same style; `build:docs` is verified against it. `pnpm audit` now exits 0, with only the pre-existing GHSA-mh99-v99m-4gvg ignore remaining. typecheck/lint/test 16/16, build:docs 9/9.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workstream 1 of 4 in the production-hardening effort. Implements
docs/superpowers/specs/2026-08-01-mock-free-test-architecture-design.md.Seven spec files that asserted against a mocked Temporal SDK now assert real effects on a real time-skipping server. A meta-test blocks reintroduction. Stryker produces a nightly mutation baseline.
Why
The unit tier reported 93.7% statement coverage — largely against a hand-written fake of the Temporal SDK. A fake agrees with our assumptions, so the test passes while real Temporal does something else.
The
isThenablebug fixed in #357 was exactly this shape: Standard Schema types the async branch asPromise<Result>, but an implementation may return anyPromiseLike. Such a value slipped pastinstanceof Promise; the code then read.issuesoff it (undefined→ "no issues, valid") and handed the handler an unvalidatedundefined. It survived a 94%-covered suite because nothing exercised the real path.That class of bug is now caught end-to-end.
handlers.contract.tsbuilds a genuine bare thenable and drives it through a real Temporal query round-trip. Zeroinstanceof Promiseguards remain inworker/contract/client.The rule
What changed
handlers.spec.tsworker.spec.tsunit, registration behavior moves toinprocessworkflow-proxy.spec.tsSTART_TO_CLOSEtimeoutcancellation.spec.tsrethrowCancellationfix, as opposite terminal statescontinue-as-new.spec.tswire-format.spec.ts"x!"not"x!!")workflow-errors.spec.tsInfrastructure: a per-path memoized bundle cache (
@temporal-contract/testing/workflow-bundle), theno-sdk-mocksguard, and nightly Stryker.Results
turbo run typecheck lint testno-sdk-mocksallowlistThe gap between 61.45% and ~94% statement coverage is the point: ~300 mutants survive — code executed by tests with nothing asserting on its behavior. One concrete case it surfaced:
errors.spec.tsasserts_tagagainst the imported constant itself, so mutating the string mutates both sides and the test can never fail.Reviewer notes
Wall-clock regressed 2.8× (13.76s → 39.09s), missing success criterion 5 as literally written. The cost is
TestWorkflowEnvironmentspawning a test-server binary (~10s per file), not bundling — the bundle cache works and is a minority of the cost. Both mitigations the spec named are implemented and effective. Accepted deliberately and recorded rather than absorbed silently. If it bites on a low-core runner, the lever ispoolOptions: { forks: { isolate: false } }on theintegration-inprocessproject.internal.ts's 33.53% mutation score is partly a measurement artifact, not test quality — the Stryker runner excludessrc/__tests__/, where all eleven new suites live. Documented inmutation.ymlso it isn't later misread as assertion rot.Every migration initially deleted effect-expressible coverage and had it restored in review — the same failure in four disguises: dropping tests as "another task covers it"; keeping a classification assertion while dropping the cause it pinned; keeping an error assertion while dropping the no-call assertion whose message was byte-identical to the receive side's; restoring one property of a pair and losing its sibling. Net coverage is now stronger; every deletion is accounted for in a per-test disposition table.
Not included, deliberately:
heartbeatTimeoutforwarding through the merge layers is asserted nowhere, and ~20% of the migrated suite is standup boilerplate awithTestWorkerhelper would absorb. Both real, neither blocking.Verification
pnpm turbo run typecheck lint test, thenpnpm turbo run test:integration --concurrency=1(Docker; the script now runs the two tiers sequentially — they previously competed inside one Vitest invocation and starvedrouting.spec.ts).