fix(principles): make the bash-mutation gate opt-in (RUN_BASH_MUTATION=1), not PATH-triggered - #1186
Merged
Merged
Conversation
…N=1), not PATH-triggered
The Stage 3C gate spawns npx vitest once per mutant, so its wall-clock
scales with the mutant count of the hook under test (13 mutants ~65s
today) and grows every time the hook is strengthened. Keyed on PATH
presence alone it fired on EVERY git push from any shell carrying
universalmutator — an unbounded, ever-growing per-push cost. Bumping the
per-test timeout would treat the symptom and keep the cost.
The gate's own docstring already declares it local/on-demand and notes it
is skipped in CI; verified 2026-07-31 that no workflow installs
universalmutator (grep over .github/workflows/*.yml -> empty), so this
changes nothing about where enforcement was ever real. It makes the
declared intent the actual behaviour instead of an accident of the
developer's PATH.
Paired evidence (both run this commit):
npx vitest run principles/02-paired-negative-test.test.ts
-> 27 passed | 1 skipped (gate correctly dormant)
RUN_BASH_MUTATION=1 npx vitest run principles/02-paired-negative-test.test.ts
-> 28 passed (gate fires, check-hook-marker.sh 100% kill, 13/13)
Prior-art: skipped — trigger-condition fix in an existing test, no new capability
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Stage 3C bash-mutation gate in
packages/core/principles/02-paired-negative-test.test.tswas triggered by tool presence on PATH (
command -v mutate && command -v analyze_mutants).Because it spawns
npx vitestonce per mutant, its wall-clock scales with the mutant count ofthe hook under test — 13 mutants ≈ 65s today — and grows every time the hook is strengthened.
Keyed on PATH alone it therefore ran on every
git pushfrom any shell carryinguniversalmutator, at an ever-growing cost.
This makes the gate opt-in:
RUN_BASH_MUTATION=1.Why not bump the per-test timeout. The observed failure was a 60s timeout against a ~65s
run, so raising the timeout makes the symptom go away and keeps the unbounded per-push cost —
the next hook strengthening puts it back over whatever number we pick. The trigger condition,
not the deadline, is the defect.
Nothing that was ever enforcing changes. The gate's own docstring already declares it
local/on-demandand states it is skipped in CI; verified on this branch that no workflowinstalls the tool:
So the gate never ran in CI, and its firing locally was an accident of the developer's PATH
rather than a designed channel. This commit makes the declared intent the actual behaviour.
Deliberate invocation is unchanged and documented in the new comment block:
Evidence (both runs on this commit)
npx vitest run principles/02-paired-negative-test.test.tsRUN_BASH_MUTATION=1 npx vitest run principles/02-paired-negative-test.test.tscheck-hook-marker.shkill rate 100% (13/13)Test Files 35 passed,Tests 340 passed | 1 skippedThe paired shape is preserved: the gate still exists, still asserts the ≥60% floor, and still
fails the push when invoked and violated — it simply no longer volunteers on every push.
§1.7 Forward-check applied
.claude/rules/rule-enforcement-channel-selection.md:26— «Choose the narrowest triggerthat still fires reliably». The check is mechanically detectable, so it stays a gate;
this PR corrects only its breadth — from "every push in a PATH-carrying shell" to
"deliberate invocation / a future scheduled job". A per-push trigger whose cost grows without
bound with every hook strengthening is not a reliable trigger.
.claude/rules/attention-is-not-a-mechanism.md:21— bare attention «may serve only asdecision/merge authority». No detection is moved onto attention here: the gate remains a
deterministic assertion with a numeric floor at
packages/core/principles/02-paired-negative-test.test.ts:327(
expect(r.status).toBe(0)against the ≥60% kill-rate wrapper). What changes is when it isinvoked, not what decides.
no-paid-llm-in-ci.md: unaffected — the gate is bash +vitest, zero API calls, and it was never in CI.
ai-laziness-traps.md §2T2 (designing ≠ running):both branches were executed on this commit and their outputs are quoted above, including the
opt-in path actually killing 13/13 mutants — not a claim that it "would still work".
§1.7 Backward-check applied
Class of this change = test-suite gates whose trigger is ambient environment probing rather
than an explicit switch. Enumerated across
packages/core/**viagrep -rn "command -v\|which \|skipIf" packages/core/principles packages/core/hooks:packages/core/principles/02-paired-negative-test.test.ts:127— the subject; GAP-FOUND,fixed here.
packages/core/principles/11-build-first-reuse-default.test.ts— Stryker gate; SWEPT:Stryker is a devDependency (installed by
npm ci, deterministic presence), not anoperator-local PATH tool, so its trigger is reproducible in CI and on every checkout — the
ambient-probe defect does not apply. No change.
.husky/pre-push:27—command -v nodecapability probe; SWEPT-CLEAN: probing for theruntime is the documented degradation path (
dual-implementation-discipline.md §4capability-check-not-brand-name), and its outcome is a fallback, not a silently-skipped
assertion.
packages/core/hooks/pre-push.ts:1570-1590—PREPUSH_ONLYseams; SWEPT-CLEAN andprecedent-setting: the file already uses explicit env switches to scope which sections run.
This PR applies the same established pattern one level down, into the principle suite, rather
than inventing a new mechanism.
No artefact is superseded; no rule previously claimed the mutation gate as a push-time channel
(
grep -rn "RUN_BASH_MUTATION\|bash mutation\|universalmutator" .claude/rules/ docs/meta-factory/returns only SSOT #91's ADAPT verdict for the mutator itself, which is about the tool, not its
trigger).
Fidelity verdict
FIDELITY: skipped — not a stage PR: there is no kickoff or spec stating an intended WHAT for this change, so there is no intent statement for a cold WHAT-conformance audit to compare the diff against. This is an operator-invited maintenance fix to a test's trigger condition, surfaced while shipping an unrelated track; its correctness is carried by the two paired runs quoted above (dormant by default, 13/13 mutants killed under the opt-in), not by design-altitude conformance.
Follow-up (not in this PR — one concern per PR)
Wiring the opt-in gate to a scheduled job (nightly
RUN_BASH_MUTATION=1over the hook suite)is the natural home for the coverage this removes from push-time. Deliberately not bundled:
it needs a workflow file, a schedule decision, and a failure-routing decision of its own.
Prior-art: skipped — trigger-condition fix in an existing test, no new capability