Skip to content

perf: traversal micro-optimizations#3

Merged
alyx merged 5 commits into
mainfrom
perf/traversal-micro
Jul 12, 2026
Merged

perf: traversal micro-optimizations#3
alyx merged 5 commits into
mainfrom
perf/traversal-micro

Conversation

@alyx

@alyx alyx commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Stacked on #2 (merge that first). Six low/medium audit findings in the traversal path, all semantics-preserving, plus two findings re-measured and deliberately skipped.

Changes (one commit each)

  • detect: marker index — markers split into literal names (map lookup) vs true globs at engine construction; *.sln-style globs reduce to HasSuffix. Walk-side detection CPU drops ~15× on marker checks. The audit prototype's shared scratch slice was redesigned into a per-hit local allocation because it would have been a data race under perf: counter fast path, single-fd reads, parallel directory walk #2's parallel walk (race detector run on all touched packages).
  • ignore: split paths once — each directory entry's rel path is split into segments once and fed to gitignore-stack, exclude/include-pattern, and smart-detect scope matching; gitignore source descriptions are no longer built when tracing is off.
  • walker: per-entry overheadpath.Join/filepath.Join → plain concatenation (inputs already clean), names slice only built when smart detection is on, trace-arg boxing guarded behind a cached flag, seen map + mutex skipped for single-root/no-symlink runs.
  • walker: lazy display paths — the display string is built only when something reads it (by-file detail, dedup, warnings, traces); aggregate runs skip ~1 string per counted file.
  • walker: tracked-dir ancestor loop — breaks at the first already-present ancestor (was O(files × depth)).

Skipped with measurements

Evidence

  • User CPU: −18.7% on the 52k-file corpus (6 interleaved pairs, low variance), −7.3% on Linux 6.9; mallocs −25.8% on Linux 6.9 aggregate.
  • Byte-identity vs perf/hot-path across: aggregate/by-file/dedup JSON, YAML, table, Linux 6.9, GOROOT/src, --tracked (plain and -vv), file/dot roots, multi-root overlap (duplicate-root trace intact), all filter flags, -v summary, full -vv on the 52k corpus. Only exception: -vvv per-file lines interleave nondeterministically on the base binary too (worker-emitted; sorted output identical).
  • go test ./... green; -race green on walker/cli/detect/ignore; cross-builds for windows/linux clean.

🤖 Generated with Claude Code

alyx and others added 5 commits July 12, 2026 10:55
Split each detector's markers at engine construction into exact names
(one map lookup per directory entry) and true globs; "*<literal>" globs
reduce to a suffix check since entry names contain no separator. Scopes
and stack entries also pre-split their directory paths once. Detector
iteration order still decides attribution when markers overlap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GitStack entries and Sets now match against pre-split segments:
IgnoredParts/IgnoredByParts/MatchedByParts let the walker split each
entry path once instead of per stack level and per pattern, and the
plain trace-off path never builds the source description. String
signatures remain as thin wrappers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Child paths are built by concatenation (inputs are clean by
construction), entry names are collected only when detection is on,
each entry's rel path is split into segments once and handed to the
segment-based matchers, hot skip sites check a cached trace flag before
building tracef arguments, and the seen map plus its lock are bypassed
when a single root without symlink following makes duplicates
impossible.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dispatched jobs carry (prefix, rel) and format the display path on
demand: aggregate runs without --by-file or --dedup never build the
per-file string, while detail, dedup ordering, warnings, and traces
still see the exact same text.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git ls-files output is grouped by directory, so most entries' ancestor
chains are already recorded; break instead of rewriting them to the
root each time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alyx
alyx deleted the branch main July 12, 2026 16:23
@alyx alyx closed this Jul 12, 2026
@alyx alyx reopened this Jul 12, 2026
@alyx
alyx changed the base branch from perf/hot-path to main July 12, 2026 16:24
@alyx
alyx merged commit a58c5ec into main Jul 12, 2026
2 checks passed
@alyx
alyx deleted the perf/traversal-micro branch July 12, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant