docs(self-scan): credit gitgalaxy for rust's confirmed macro-body class/function wins - #1941
Merged
Merged
Conversation
…ss/function wins
Both rust ledger entries below were already validated with a clear verdict
confirming GitGalaxy's solo claim is real, and ctags+tree-sitter's
non-corroboration is a confirmed, structural limitation in them (both
treat macro_rules!/quote!{} bodies as opaque token trees and can't emit
struct/fn nodes from inside one) -- but credit_tools was never set when
they were validated, so the confirmed-correct occurrences never moved
GitGalaxy's actual precision number, only suppressed the chart's `*`.
Same gap class as the 2026-08-20 C precision fix documented in
tri_comparison_ledger.py's own VERIFIED ADJUSTMENTS section.
- rust/function/existence/agree[gitgalaxy]_vs[ctags,tree_sitter] (152
occurrences, fn definitions inside quote!{} proc-macro bodies)
- rust/class/existence/agree[gitgalaxy]_vs[ctags,tree_sitter] (25
occurrences, struct definitions inside macro_rules! bodies)
Both now credit_tools: ["gitgalaxy"]. Verified the delta directly before
regenerating: gitgalaxy func precision 1775/1927 (92.1%) -> 1927/1927
(100.0%), class precision 287/312 (92.0%) -> 312/312 (100.0%), exactly
matching the 152/25 occurrence counts. Full `--all --write` regen confirms
scope: diffed against committed HEAD, only these two entries' credit_tools
field changed anywhere in the ledger (no new/removed keys, no other
language's status/verdict/investigated_by/credit/debit touched), and the
SVG diff is exactly the two rust precision bars going full-width.
Also swept the rest of the ledger for the same missed-credit pattern
(validated solo-claim shapes with empty credit_tools) across all
languages: 12 candidates found, 10 correctly need no adjustment (either
the solo tool's claim was confirmed WRONG, already correctly unrewarded
by default, or a genuinely mixed shape where "leave empty" is the right
call per the module's own criteria) -- rust's two were the only clean
"one tool confirmed right, others' silence structurally explained" cases
in the whole ledger.
Verified: ruff_audit.py --ci and mypy_audit.py --ci both clean (no new
findings beyond baseline).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
squid-protocol
added a commit
that referenced
this pull request
Aug 20, 2026
…#1943) _winner_or_tie() returned "tie" (no badge) whenever 2+ tools shared the top rate_pct, with no tie-break at all. Real case this misses: rust's Func/Class Precision now ties 3-way at 100% (GitGalaxy 1927/1927, tree-sitter 1775/1775, ctags 1774/1774) once GitGalaxy's macro-body-only claims are ledger-validated (#1941) -- each tool is simply never wrong about what it itself claims, at very different claim counts, so a rate-only comparison awards nobody despite GitGalaxy having demonstrably found more of the validated-real total. Break a rate tie using each tied tool's absolute matched_consensus (its count of validated-correct occurrences), not the raw rate alone. This is the same "more evidence should count for something" principle as the existing sample-size fix (a 2-sample 100% cell no longer silently outranks an 80-sample 98.75% cell), just pointed the other direction: a 1927-sample validated 100% shouldn't lose a badge to a 1774-sample validated 100% just because both cleared the same bar. Only ever reached after the caller's has_open_question() check, so every count used here is already either unquestioned or ledger-validated -- not a reversion to the "just trust the bigger number" anti-pattern this system exists to prevent. Regenerated the full 45-language chart: rust now shows a real GitGalaxy badge on both Func and Class Precision (summary tally 4->6 languages, ties 18->16); no other language's badges changed. Verified the tie logic directly against rust's real numbers plus two guard cases: a genuine unbreakable tie (equal rate AND equal count) still returns "tie", and the original sample-size-bug scenario (2/2 vs. 79/80) is unaffected since it was never a tie to begin with. Also adds a new CLAUDE.md section generalizing "badges require verification, not just the higher number" (previously only documented inside tri_comparison_chart.py's own docstring and the tri-comparison-ledger-sweep skill) into a repo-wide principle for any comparative-correctness claim, plus the new tie-break rule. Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
credit_toolswas never set, so the confirmed-correct occurrences never moved GitGalaxy's actual precision number, only suppressed the chart's*. Same gap class as the 2026-08-20 C precision fix already documented intri_comparison_ledger.py's VERIFIED ADJUSTMENTS section.rust/function/existence/agree[gitgalaxy]_vs[ctags,tree_sitter](152 occurrences --fndefinitions insidequote!{}proc-macro bodies)rust/class/existence/agree[gitgalaxy]_vs[ctags,tree_sitter](25 occurrences --structdefinitions insidemacro_rules!bodies)credit_tools: ["gitgalaxy"]. Verified the exact delta before regenerating: gitgalaxy func precision 1775/1927 (92.1%) -> 1927/1927 (100.0%), class precision 287/312 (92.0%) -> 312/312 (100.0%).tri_comparison_chart.py --all --writeregen and diffed against committed HEAD to confirm scope: only these two entries'credit_toolsfield changed anywhere in the ledger (no new/removed keys, no other language'sstatus/verdict/investigated_by/credit/debit touched), and the SVG diff is exactly the two rust precision bars going full-width.credit_toolsfound, 10 correctly need no adjustment (solo tool's claim confirmed wrong and already unrewarded by default, or a genuinely mixed shape where "leave empty" is correct) -- rust's two were the only clean "one tool confirmed right, others structurally can't corroborate" cases in the whole ledger.Test plan
python tests/ruff_audit.py --ci-- clean, no new findingspython tests/mypy_audit.py --ci-- clean, no new findingsmatched_consensusdelta (152/25) matches occurrence counts exactly before trusting the regenerated chart🤖 Generated with Claude Code