ci(mutants): expand mutation testing scope to dispatch + classify, enable on PRs#320
Merged
Destynova2 merged 3 commits intomainfrom Apr 28, 2026
Merged
ci(mutants): expand mutation testing scope to dispatch + classify, enable on PRs#320Destynova2 merged 3 commits intomainfrom
Destynova2 merged 3 commits intomainfrom
Conversation
…able on PRs Adds mutation testing for `src/server/dispatch/` (4 shards) and `src/routing/classify/` (2 shards) to the existing main-only matrix and introduces a PR-triggered `mutants-pr` job that runs only on the Rust files the PR touches within the curated scope (router, dispatch, classify, dlp). The PR job wall-clocks at 25 min via a wrapper script and writes a sticky comment + job summary with caught/missed counts; it stays informational and never blocks merge — the full matrix on `main` remains the source of truth. Files: - `.github/workflows/ci.yml`: 6 new shards on `mutants` (5a/b/c/d for dispatch, 6a/b for classify), new `mutants-pr` job, no change to the `required` gate (mutation testing stays informational). - `scripts/mutation-pr.sh`: diff-based filtering against the PR base ref, scope filter (router/dispatch/classify/dlp), 25 min wall-clock cap, jq summary parser, GITHUB_OUTPUT integration. `.cargo/mutants.toml` is intentionally left unchanged: the existing five exclusions already document each unreachable mutant with a file/line rationale, and tightening the list is a separate, code-touching effort.
added 2 commits
April 28, 2026 22:44
…C2016
actionlint's shellcheck integration flagged the prior `printf '%s\n...'`
single-quoted format string with SC2016 (backtick characters in markdown
backticks misread as command substitution candidates). Replace the
printf with `{ echo ... } >file` and feed `gh pr comment --body-file`
plus `gh api --method PATCH` via `jq -Rs '{body: .}'`. Same external
behaviour, no more SC2016 noise.
Backticks in echos that wrap shell variables can be parsed by shellcheck as command substitution candidates — even when escaped — and the actionlint+shellcheck combo running in CI flagged them. Replace with plain ASCII delimiters in the job-summary output and the comment-body echos.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/server/dispatch/(4 shards: mod.rs split 0/2 + 1/2, retry.rs, provider_loop.rs) andsrc/routing/classify/(2 shards: mod.rs, classify.rs) to the existing main-only matrix.mutants-prjob that runs only on the Rust files the PR touches within the curated scope (router, dispatch, classify, dlp). Wall-clocks at 25 min, writes a job summary + sticky PR comment, and stays informational — the full matrix onmainremains the source of truth.scripts/mutation-pr.sh(diff-based filtering, scope filter, jq summary parsing, GITHUB_OUTPUT wiring).Why
mainand only coveredrouter/+features/dlp/(5 files). Critical untested modules:server/dispatch/,routing/classify/.main, where the previous setup just produced post-merge noise.Scope decisions
src/server/dispatch/mod.rs(575 LoC)--shard 0/2and1/2)pii.rssrc/server/dispatch/retry.rs(478 LoC)src/server/dispatch/provider_loop.rs(275 LoC)src/routing/classify/mod.rs(476 LoC)src/routing/classify/classify.rs(512 LoC)resolver.rs,telemetry.rs,inference.rs,tier_match.rs, etc. were not added to the matrix yet to keep total per-PR cost bounded; they can land in a follow-up.PR sampling strategy (
mutants-pr)BASE...HEADto find changedsrc/**/*.rsfiles.status=skipped-out-of-scope).cargo mutants --file ... -- --libwithtimeout --foreground --preserve-status 1500(25 min cap).mutants.out/outcomes.jsonviajqfor caught/missed/timeout/unviable counts.GITHUB_STEP_SUMMARY+ upserts a single PR comment (no spam on re-runs).Status legend exposed to the PR comment:
clean,missed,timed-out,skipped-no-rust,skipped-out-of-scope..cargo/mutants.tomlLeft unchanged. The existing five exclusions already document each unreachable mutant with a file/line rationale; tightening the regex list is a separate, code-touching effort and was deferred to keep this PR diff scoped to CI infrastructure.
Caveat
If the new
mutants-prjob (or the next main run with the expanded matrix) surfaces a wave of surviving mutations ondispatch/— likely, since the audit found 0 unit tests there — those are tracked as test gaps for the paralleltest/dispatch-unit-testsPR. This PR explicitly does not aim to fix them.This PR could not run cargo-mutants locally to produce concrete survivor counts (the Claude Code sandbox blocks
cargoexecution). The first run of the workflow on this branch will produce the baseline; the comment it posts becomes the acceptance criteria for the unit-test PR.Test plan
mutants-prjob runs (this PR touches.github/workflows/ci.yml+scripts/, both outside the scope filter, so the script should exitskipped-out-of-scope→ no mutation work performed).mutantsshards 1-4 still green on the next main push.continue-on-error: true).validate-yaml(actionlint) passes on the new YAML.requiredgate is unchanged — neithermutantsnormutants-prblocks merge.🤖 Generated with Claude Code