docs(mutation) #41: record the first completed sweep - 83%, 22 minutes - #123
Conversation
The offsets.* sweep completes in 21m55s and scores 83% (185 mutants, 153 killed, 92% test strength). The internal.* sweep it replaced never finished, in 42+ minutes on CI or 83+ locally, so this is the first mutation number this project has ever had. Recorded in the repo because nothing else keeps it. The PIT report is a 14-day artifact and the statistics exist only in a job log - which ages out, and which a re-run silently replaces, as I proved earlier this week by re-running a job whose logs I had not saved. A baseline nobody can find is not a baseline. Two decisions the number settles rather than merely informs: - The sweep is schedulable. `push: branches: [master]` was left unwired pending a runtime, and 22 minutes on the highcpu box is affordable. The note now says so instead of treating it as an open question. - Narrowing target-tests is NOT worth doing. Coverage is 311s of 1315s, about 24%, so the accuracy it costs - mutants killed outside offsets.* reported as no-coverage - buys little. That had been carried as a live option. It also makes the marginal cost of widening PIT_DECIDABLE_PACKAGES explicit: the coverage pass is paid whatever the target, so adding state. costs only its own mutants. Replaces the "do not quote a mutation score until a sweep completes" line, which was correct when written and is now the opposite of the truth. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
|
Claude finished @astubbs's task in 1m 21s —— View job Claude finished the reviewTodo list
ReviewDoc-only change, no code/tests affected. The numbers all check out internally — I re-derived each one instead of trusting the prose:
One real issue: Contradiction left in the "give it a trigger" bullet ( Minor/non-blocking:
Everything else — the "schedulable" and "narrowing |
✅ Duplicate Code ReportTwo engines run in parallel for cross-validation. Each has its own thresholds tuned to its baseline - the real safety net is the per-engine "max increase vs base" check. ✅ PMD CPD
No new clones introduced by this PR. ✅ jscpd (language-agnostic)
No new clones introduced by this PR. Powered by astubbs/duplicate-code-cross-check |
✅ SpotBugs ReportNo bugs found (new bugs only — baseline from base branch excluded). |
The house convention is that a bare #NN means THIS fork and upstream is written `upstream #NN`. Bare `confluentinc#859` in source was merely ambiguous before; now that the mirror #120 exists - and fork confluentinc#859 will eventually exist and mean something else - it is wrong. 27 refs qualified across the Java sources and docs/refactoring.md this PR already touches: #200, #233, confluentinc#857, confluentinc#859, confluentinc#893, confluentinc#905. Comments and one assertion message only; no logic. This is the "correct opportunistically as files are touched" rule from #114's Phase 4, not the bulk rewrite that plan rejects. Deliberately left alone, because a mechanical sweep gets them wrong: - CHANGELOG.adoc - its own header states that entries below 0.6.0.0 predate the fork and their #NN already refer to upstream, and the 0.6.0.0 entries use fully qualified links. Rewriting either would be wrong. - upstream-map.yaml - every number in it is upstream by construction, so prefixing all of them is noise, not clarity. - AGENTS.md - the only hit is a commit-subject convention shown BY EXAMPLE; rewriting the example would silently change the documented convention. Note what this does NOT do: renumber references to fork issue numbers. #114 rejects that, and three of its four reasons still stand - git history is immutable, a mechanical remap cannot tell an issue from the #999/#123 test offsets in Java source, and fork numbering will eventually collide with upstream's range. The fourth reason, staying cherry-pickable upstream, no longer applies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
astubbs
changed the title
astubbs
changed the title
|
Thanks - the contradiction was real and is fixed in 8b2eea5, along with two more instances of it that the review didn't reach. The flagged bullet (
The long line. Rewrapped to the file's ~103-char width. It was 200. Two more places said the same stale thing, both worse than the one flagged because they're read before the ledger is:
Also in scope now, both from following up on the issue tracking rather than the review: #41 is reopened with a status comment (its ask did land - the sweep runs on On the arithmetic re-derivation - that's the useful half of this review. It's what would catch a transcription slip between the job log and the table, which is the actual failure mode here given the log ages out. |
The first sweep completed, but three places still told a future reader the runtime was unknown - including the mutation-full-sweep.yml header block, which is the first thing anyone reads before dispatching, and which was headed UNMEASURED. - docs/inflight/ci-mutation-testing.md: one bullet opened "now unblocked, at a known 22 minutes per run" and closed "Deliberately NOT wired until the runtime is known". It still is not wired, so that fact stays; the stale reason goes. - docs/plans/2026-08-03-002-mutation-testing-plan.md: "Do not quote a mutation score for this project until one has" - correct when written, now the opposite of the truth. Points at the baseline instead. - .github/workflows/mutation-full-sweep.yml: comments only - no trigger, runner, permission or step change. Also corrects the coverage pass from an estimated ~332s to the measured 311s, and drops the "until this merges, dispatch fails" note that #111 made obsolete. Also fixes the convention that made this PR's own title collide. AGENTS.md and .gitmessage both taught a trailing "(#NNN)" issue ref, but GitHub appends the PR number to exactly that slot on squash-merge, so a title ending "(#41)" merges as "(#41) (#123)" - two bare numbers with nothing to tell them apart. Both now put a fork issue at the front, as this commit does, matching Apache Kafka and our own pre-fork confluentincGH-725 history. Upstream refs stay inline and worded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @astubbs's task in 1m 59s —— View job ReviewTodo list
The three fixes the author listed in the last comment all check out against the actual diff:
One real issue, same failure mode as the one already fixed, in the same file:
This is the sentence right at the top of the file — the first thing a reader sees, before the Also verified, no issues found:
|
The first sweep completed, but three places still told a future reader the runtime was unknown - including the mutation-full-sweep.yml header block, which is the first thing anyone reads before dispatching, and which was headed UNMEASURED. - docs/inflight/ci-mutation-testing.md: one bullet opened "now unblocked, at a known 22 minutes per run" and closed "Deliberately NOT wired until the runtime is known". It still is not wired, so that fact stays; the stale reason goes. - docs/plans/2026-08-03-002-mutation-testing-plan.md: "Do not quote a mutation score for this project until one has" - correct when written, now the opposite of the truth. Points at the baseline instead. - .github/workflows/mutation-full-sweep.yml: comments only - no trigger, runner, permission or step change. Also corrects the coverage pass from an estimated ~332s to the measured 311s, and drops the "until this merges, dispatch fails" note that #111 made obsolete. Also fixes the convention that made this PR's own title collide. AGENTS.md and .gitmessage both taught a trailing "(#NNN)" issue ref, but GitHub appends the PR number to exactly that slot on squash-merge, so a title ending "(#41)" merges as "(#41) (#123)" - two bare numbers with nothing to tell them apart. Issue refs now go at the FRONT, as this commit's subject does, matching Apache Kafka and our own pre-fork confluentincGH-725 history. That holds for upstream refs too: they just need the word, as in "fix(core) upstream confluentinc#909: subject", because a bare "confluentinc#909" autolinks to FORK issue 909 rather than the upstream one meant. Where a fork mirror exists, prefer its number (#119 mirrors upstream confluentinc#857) and leave the upstream number to the Upstream-Issue trailer, which is what tooling reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Amended and force-pushed: the SHA cited above is now fc1d2e4 ( One substantive change on top of it. The convention I added had a carve-out saying upstream refs could stay inline at the end of the subject - which was rationalising the existing The word is not only for humans - a bare |
8b2eea5 to
fc1d2e4
Compare
|
Claude finished @astubbs's task in 3m 1s —— View job ReviewTodo list
The three fixes named in the author's second follow-up comment (contradiction removal, header rewrite, plan-doc paragraph) are all present and correct in the current HEAD ( One real issue survives, and it's the exact failure mode already flagged twice on this PR — same file, four lines above where the last review looked:
Second issue, new in this round — a residual contradiction in the same file the "Rider" section just rewrote:
This tells the reader to "keep" a subject convention whose own examples ( [Fix this →](https://claude.ai/code?q=In%20AGENTS.md%20on%20branch%20docs%2Fmutation-baseline%2C%20line%20312%20(the%20%22Commit%20trailers%22%20section%2C%20near%20%22Enable%20the%20editor%20prompt%20once%20per%20checkout%22)%20says%20%22Keep%20the%20existing%20subject%20convention%20for%20upstream%20references%20(...%20(%23893)%2C%20cherry-pick%20Confluent%20%23905)%22%20then%20immediately%20says%20the%20trailing%20(%23N)%20slot%20belongs%20to%20the%20PR%20number%20-%20contradicting%20itself%2C%20the%20same%20bug%20the%20PR%20Discipline%20bullet%20at%20line%20234%20was%20rewritten%20to%20fix.%20Reword%20it%20to%20stop%20telling%20readers%20to%20keep%20a%20trailing-(%23893)-style%20convention%2C%20consistent%20with%20line%20234%27s%20%22fix(core)%20upstream%20%23909%3A%20subject%22%20/ %20%22GH-725%3A%20...%20(%23727)%22%20examples.&repo=astubbs/parallel-consumer) Also verified, no issues found:
|
…ach the gate about anchors Two things the reference sweep surfaced in Java sources. The quarantine script tests built their fixture registries with "Owner: PR #999", "PR #80" and "PR #123". The last two are real fork PRs, so the fixtures read as genuine references to anyone grepping, and #999 is close enough to the live range to be mistaken for one. They are now #999999 and #999998 - unmistakably fake, and above the threshold where a bare number is ambiguous. Two distinct values, because the owner-mismatch tests compare a @Quarantined value against a registry value and need them to differ. 27 tests pass. The gate also flagged references that were already qualified by an html anchor - TransactionMarkersTest has <a href="...confluentinc/issues/329">Github issue confluentinc#329</a>, where the href names the repo and the number in the link text is unambiguous to any reader. stripQualified removed bare URLs but not the anchor element, so the visible "confluentinc#329" looked bare. It now strips the whole element, with a test confirming a bare number elsewhere on the same line is still caught. Together these take the tree-wide backlog from 390 to 374, and the Java share from 53 to 36. The follow-up note now carries the finished classification of all 36 Java references - file, line, and which repo each means, resolved against both - so that pass does not repeat this research. Also records that @tag("confluentinc#355") is a tag-string rename rather than a prose prefix, and that nothing selects on that tag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
…r repo Mirrors all 78 open issues from confluentinc/parallel-consumer into this fork (#44, #117-#195, label upstream-mirror), each carrying a code-backed diagnosis, and each backlinked from its upstream original while that is still possible - archival kills writes, not reads, so the backlinks were the half with a deadline. Seven are closed against a released version. Everything else here follows from that. Mirroring made bare issue numbers ambiguous. The fork numbers from 1 and confluentinc reaches confluentinc#922, so the ranges overlap completely: of the 51 numbers cited across the files this touches, 48 exist in BOTH repos meaning different things. #29 is our rebalance fix and confluentinc#29 is an async-sending request; #114 is a docs PR and confluentinc#114 is a GPG key issue. So a reference now names its repo below #1000, and a CI gate enforces it on added lines. The gate went through three designs, and the discarded two look plausible enough to be worth recording. Comparing against "the fork is at #N" raced - CI read 196 while #197 already existed. Checking whether a number resolves here fails worse: `#200` resolves, to a fork issue about ManagedTruth, while the author meant confluentinc#200, shared-nothing architecture. A wrong reference that resolves is worse than a broken one, because nothing looks amiss. The rule is textual, so it makes no API calls and cannot race. The qualifier names the owner rather than the role - confluentinc#857, not "upstream confluentinc#857". "Upstream" describes a relationship and is not stable: this repo is upstream to anyone who forks it. "Fork" is out for the same reason. Also swept every reference in the files touched here, fixed the source comments behind the generated TODO index rather than the index, and stopped the quarantine fixtures borrowing real PR numbers - #80 and #123 are live fork PRs, so the fixtures read as genuine references. The map shrinks to match: upstream-map.yaml tracks upstream PRs only, because issues now live in the mirror, and the manifest-driven backlink tooling is retired - it commented one issue per map entry, and the map no longer holds issues. Two corrections the work surfaced: the sweep was reporting our own backlink comments as upstream activity, hiding the two real replies among all 78; and confluentinc#548 was recorded open when it merged in 2023 and is already in this fork. Remaining tree-wide references are deliberately out of scope, tracked in docs/inflight/next-qualify-remaining-refs.md with the Java set already classified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
…r repo Mirrors all 78 open issues from confluentinc/parallel-consumer into this fork (#44, #117-#195, label upstream-mirror), each carrying a code-backed diagnosis, and each backlinked from its upstream original while that is still possible - archival kills writes, not reads, so the backlinks were the half with a deadline. Seven are closed against a released version. Everything else here follows from that. Mirroring made bare issue numbers ambiguous. The fork numbers from 1 and confluentinc reaches confluentinc#922, so the ranges overlap completely: of the 51 numbers cited across the files this touches, 48 exist in BOTH repos meaning different things. #29 is our rebalance fix and confluentinc#29 is an async-sending request; #114 is a docs PR and confluentinc#114 is a GPG key issue. So a reference now names its repo below #1000, and a CI gate enforces it on added lines. The gate went through three designs, and the discarded two look plausible enough to be worth recording. Comparing against "the fork is at #N" raced - CI read 196 while #197 already existed. Checking whether a number resolves here fails worse: `#200` resolves, to a fork issue about ManagedTruth, while the author meant confluentinc#200, shared-nothing architecture. A wrong reference that resolves is worse than a broken one, because nothing looks amiss. The rule is textual, so it makes no API calls and cannot race. The qualifier names the owner rather than the role - confluentinc#857, not "upstream confluentinc#857". "Upstream" describes a relationship and is not stable: this repo is upstream to anyone who forks it. "Fork" is out for the same reason. Also swept every reference in the files touched here, fixed the source comments behind the generated TODO index rather than the index, and stopped the quarantine fixtures borrowing real PR numbers - #80 and #123 are live fork PRs, so the fixtures read as genuine references. The map shrinks to match: upstream-map.yaml tracks upstream PRs only, because issues now live in the mirror, and the manifest-driven backlink tooling is retired - it commented one issue per map entry, and the map no longer holds issues. Two corrections the work surfaced: the sweep was reporting our own backlink comments as upstream activity, hiding the two real replies among all 78; and confluentinc#548 was recorded open when it merged in 2023 and is already in this fork. Remaining tree-wide references are deliberately out of scope, tracked in docs/inflight/next-qualify-remaining-refs.md with the Java set already classified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
…r repo Mirrors all 78 open issues from confluentinc/parallel-consumer into this fork (#44, #117-#195, label upstream-mirror), each carrying a code-backed diagnosis, and each backlinked from its upstream original while that is still possible - archival kills writes, not reads, so the backlinks were the half with a deadline. Seven are closed against a released version. Everything else here follows from that. Mirroring made bare issue numbers ambiguous. The fork numbers from 1 and confluentinc reaches confluentinc#922, so the ranges overlap completely: of the 51 numbers cited across the files this touches, 48 exist in BOTH repos meaning different things. #29 is our rebalance fix and confluentinc#29 is an async-sending request; #114 is a docs PR and confluentinc#114 is a GPG key issue. So a reference now names its repo below #1000, and a CI gate enforces it on added lines. The gate went through three designs, and the discarded two look plausible enough to be worth recording. Comparing against "the fork is at #N" raced - CI read 196 while #197 already existed. Checking whether a number resolves here fails worse: `#200` resolves, to a fork issue about ManagedTruth, while the author meant confluentinc#200, shared-nothing architecture. A wrong reference that resolves is worse than a broken one, because nothing looks amiss. The rule is textual, so it makes no API calls and cannot race. The qualifier names the owner rather than the role - confluentinc#857, not "upstream confluentinc#857". "Upstream" describes a relationship and is not stable: this repo is upstream to anyone who forks it. "Fork" is out for the same reason. Also swept every reference in the files touched here, fixed the source comments behind the generated TODO index rather than the index, and stopped the quarantine fixtures borrowing real PR numbers - #80 and #123 are live fork PRs, so the fixtures read as genuine references. The map shrinks to match: upstream-map.yaml tracks upstream PRs only, because issues now live in the mirror, and the manifest-driven backlink tooling is retired - it commented one issue per map entry, and the map no longer holds issues. Two corrections the work surfaced: the sweep was reporting our own backlink comments as upstream activity, hiding the two real replies among all 78; and confluentinc#548 was recorded open when it merged in 2023 and is already in this fork. Remaining tree-wide references are deliberately out of scope, tracked in docs/inflight/next-qualify-remaining-refs.md with the Java set already classified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
…r repo Mirrors all 78 open issues from confluentinc/parallel-consumer into this fork (#44, #117-#195, label upstream-mirror), each carrying a code-backed diagnosis, and each backlinked from its upstream original while that is still possible - archival kills writes, not reads, so the backlinks were the half with a deadline. Seven are closed against a released version. Everything else here follows from that. Mirroring made bare issue numbers ambiguous. The fork numbers from 1 and confluentinc reaches confluentinc#922, so the ranges overlap completely: of the 51 numbers cited across the files this touches, 48 exist in BOTH repos meaning different things. #29 is our rebalance fix and confluentinc#29 is an async-sending request; #114 is a docs PR and confluentinc#114 is a GPG key issue. So a reference now names its repo below #1000, and a CI gate enforces it on added lines. The gate went through three designs, and the discarded two look plausible enough to be worth recording. Comparing against "the fork is at #N" raced - CI read 196 while #197 already existed. Checking whether a number resolves here fails worse: `#200` resolves, to a fork issue about ManagedTruth, while the author meant confluentinc#200, shared-nothing architecture. A wrong reference that resolves is worse than a broken one, because nothing looks amiss. The rule is textual, so it makes no API calls and cannot race. The qualifier names the owner rather than the role - confluentinc#857, not "upstream confluentinc#857". "Upstream" describes a relationship and is not stable: this repo is upstream to anyone who forks it. "Fork" is out for the same reason. Also swept every reference in the files touched here, fixed the source comments behind the generated TODO index rather than the index, and stopped the quarantine fixtures borrowing real PR numbers - #80 and #123 are live fork PRs, so the fixtures read as genuine references. The map shrinks to match: upstream-map.yaml tracks upstream PRs only, because issues now live in the mirror, and the manifest-driven backlink tooling is retired - it commented one issue per map entry, and the map no longer holds issues. Two corrections the work surfaced: the sweep was reporting our own backlink comments as upstream activity, hiding the two real replies among all 78; and confluentinc#548 was recorded open when it merged in 2023 and is already in this fork. Remaining tree-wide references are deliberately out of scope, tracked in docs/inflight/next-qualify-remaining-refs.md with the Java set already classified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
1 participant
Description
The first
offsets.*mutation sweep completed - 21m55s, 83% score. Theinternal.*sweep it replaced never finished, in 42+ minutes on CI or 83+ locally, so this is the first mutation number this project has ever had.This records it in the repo, because nothing else keeps it: the PIT report is a 14-day artifact, and the statistics live only in a job log - which ages out, and which a re-run silently replaces (demonstrated the hard way earlier this week, re-running a job whose logs hadn't been saved). A baseline nobody can find is not a baseline.
Tracks #41, deliberately without closing it. That issue asked whether PIT could work on the self-hosted runner; the baseline answers it - yes, in 22 minutes - but the re-widening work behind it continues, so it is reopened and carries the remaining items.
mutation-full-sweep#1, against master at58991506Two decisions it settles
Both were left open in #111 pending exactly this measurement, so this closes them with evidence rather than argument.
The sweep is schedulable.
push: branches: [master]was deliberately unwired because nothing had measured the runtime. 22 minutes on the highcpu box is affordable, so that is now a cost question with a real number.Narrowing
target-testsis not worth doing. Coverage is 311s of 1315s - about 24% - so the accuracy it would cost (mutants killed outsideoffsets.*reported as no-coverage) buys little. That had been carried as a live option since #111.It also makes the marginal cost of widening
PIT_DECIDABLE_PACKAGESexplicit: the coverage pass is paid whatever the target, so addingstate.costs only its own mutants.What it replaces
Three places still told a future reader that the runtime was unknown. All three now agree with the measurement:
docs/inflight/ci-mutation-testing.md- the re-widen list ended a bullet with "Deliberately NOT wired until the runtime is known", in the same paragraph that now opens "now unblocked, at a known 22 minutes per run". It still isn't wired, so that fact stays; the stale reason is gone.docs/plans/2026-08-03-002-mutation-testing-plan.md- "Do not quote a mutation score for this project until one has [completed]", correct when written and now the opposite of the truth. It points at the baseline instead..github/workflows/mutation-full-sweep.yml- the header block, which is the first thing anyone reads before dispatching, was headed UNMEASURED and opened "nothing has completed here YET". Also corrects the coverage pass from an estimated ~332s to the measured 311s, and drops the "until this merges, dispatch fails" note that ci(mutation): point PIT where survivors are real, and make the lane trustworthy #111 made obsolete.Rider: the trailing
(#N)slot belongs to the PR numberFound while citing #41 in this PR's own title.
AGENTS.mdsaid "keep the existing subject convention (... (#893))" and.gitmessagesaid "reference the upstream issue/PR in the subject as(#NNN)" - both of which walk you straight into a collision, because GitHub appends the PR number to that exact slot on squash-merge. A title ending(#41)merges as(#41) (#123): two bare numbers, nothing to distinguish issue from PR.Both now say the same thing instead: the end of the subject is the PR number's, a fork issue goes at the FRONT as
type(scope) #41: subject, and upstream refs stay inline and worded ((upstream #912),(#857 family)). That matches Apache Kafka (KAFKA-14561: ... (#13114)) and this repo's own pre-fork history (GH-725: ... (#727)). This PR's title is the first to follow it.Nothing enforces titles, so this is documentation, not a gate.
Checklist
mutation-full-sweep.ymlis edited, but comments only: no trigger, runner, permission, or step change. Verified by diffing out comment lines (empty) and re-parsing the YAML.🤖 Generated with Claude Code