counter: fuse the per-line loop into one whole-buffer scan#8
Merged
Conversation
Count previously touched every line three ways: an IndexByte call to carve the line out, a TrimSpace scan to test blankness, and the classify call that scanned it again. The fused scan makes line boundaries events inside one state machine driven by a single per-byte class table: - plain code and whitespace skip in run loops (one load per byte); blankness falls out of per-line flags, so TrimSpace runs only for lines whose bytes it might strip (\v, \f, non-ASCII, interior \r) - multi-line strings and non-nested block comments locate their closer with one bytes.Index over the remaining buffer instead of one failed search per line, then replay the enclosed line boundaries - the normal-state scan is an inner loop entered per state change, so the multi-line state checks no longer run per byte - a cached next-newline position serves every string and line-comment marker on the same line Semantics are preserved exactly: blank-beats-everything via TrimSpace (including Unicode spaces), one stripped trailing \r, line-bounded single-line strings, escape parity that cannot cross newlines. Verified against oldCount over GOROOT/src (9,992 files, 0 mismatches) plus hardened edge-case and random-soup differentials (\v/\f/NBSP blanks, interior \r, CRLF multi-line strings, blanks inside block comments). Best-of-6 same-process benchmarks vs the per-line gate-table scan: GoServer 853->952 MB/s (+12%), GoZerrors 485->764 (+58%), JavaScript 340->376 (+11%), Python 378->434 (+15%). End-to-end output on GOROOT/src is byte-identical for default, --dedup --hidden, and --by-file runs. prev_impl_test.go keeps the replaced implementation for benchmarking, mirroring old_impl_test.go. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces
Count's outer line-splitting loop + per-lineclassifywith a single fused pass over the whole buffer, driven by a per-byte class table (fclass):bytes.TrimSpaceonly runs for lines containing bytes it might strip (\v,\f, non-ASCII, interior\r).bytes.Indexover the remaining buffer instead of one failed search per line, then replay the enclosed line boundaries.Semantics
Byte-identical to the previous scanner, including the corners: blank-beats-everything via TrimSpace (Unicode spaces included), exactly one stripped trailing
\r, line-bounded single-line strings, escape parity that cannot cross newlines.Verification:
oldCountover GOROOT/src: 9,992 files, 0 mismatches\v/\f/NBSP blanks, interior\r, CRLF multi-line strings, blanks inside block comments--dedup --hidden, and--by-filerunsPerformance
Best-of-6, same-process, interleaved vs the per-line gate-table scan (Apple M5):
Wall time on tree scans is unchanged (syscall-bound; see #6/#7); user CPU drops ~8%. The fused, event-driven loop is also the stage-2 substrate a future SIMD structural-indexing pass (bitmask stage 1) would plug into.
prev_impl_test.gokeeps the replaced implementation as an in-process bench baseline, mirroringold_impl_test.go.🤖 Generated with Claude Code