perf: traversal micro-optimizations#3
Merged
Merged
Conversation
Split each detector's markers at engine construction into exact names (one map lookup per directory entry) and true globs; "*<literal>" globs reduce to a suffix check since entry names contain no separator. Scopes and stack entries also pre-split their directory paths once. Detector iteration order still decides attribution when markers overlap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GitStack entries and Sets now match against pre-split segments: IgnoredParts/IgnoredByParts/MatchedByParts let the walker split each entry path once instead of per stack level and per pattern, and the plain trace-off path never builds the source description. String signatures remain as thin wrappers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Child paths are built by concatenation (inputs are clean by construction), entry names are collected only when detection is on, each entry's rel path is split into segments once and handed to the segment-based matchers, hot skip sites check a cached trace flag before building tracef arguments, and the seen map plus its lock are bypassed when a single root without symlink following makes duplicates impossible. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dispatched jobs carry (prefix, rel) and format the display path on demand: aggregate runs without --by-file or --dedup never build the per-file string, while detail, dedup ordering, warnings, and traces still see the exact same text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git ls-files output is grouped by directory, so most entries' ancestor chains are already recorded; break instead of rewriting them to the root each time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2 (merge that first). Six low/medium audit findings in the traversal path, all semantics-preserving, plus two findings re-measured and deliberately skipped.
Changes (one commit each)
*.sln-style globs reduce toHasSuffix. Walk-side detection CPU drops ~15× on marker checks. The audit prototype's shared scratch slice was redesigned into a per-hit local allocation because it would have been a data race under perf: counter fast path, single-fd reads, parallel directory walk #2's parallel walk (race detector run on all touched packages).path.Join/filepath.Join→ plain concatenation (inputs already clean),namesslice only built when smart detection is on, trace-arg boxing guarded behind a cached flag,seenmap + mutex skipped for single-root/no-symlink runs.Skipped with measurements
-jobs 8/-jobs 4are indistinguishable in wall time — the win the audit measured predated the parallel walk. Not applied.GOGC=800is now ~1% user CPU (below the 3% bar) — perf: counter fast path, single-fd reads, parallel directory walk #2's pooled reads removed the garbage that made it pay. Not applied.Evidence
perf/hot-pathacross: aggregate/by-file/dedup JSON, YAML, table, Linux 6.9, GOROOT/src,--tracked(plain and-vv), file/dot roots, multi-root overlap (duplicate-root trace intact), all filter flags,-vsummary, full-vvon the 52k corpus. Only exception:-vvvper-file lines interleave nondeterministically on the base binary too (worker-emitted; sorted output identical).go test ./...green;-racegreen on walker/cli/detect/ignore; cross-builds for windows/linux clean.🤖 Generated with Claude Code