scripts: consolidate benchmarking into bench.py; record scan/hash & walker dead-ends

[martinus]

What

Replace the three overlapping bench-*.sh scripts with a single extensible Python harness, scripts/bench.py, and document two measured dead-ends so they aren't re-attempted.

scripts/bench.py — one benchmark harness

Subsumes bench-scan.sh (warm A/B scan-makespan), bench-scan-cold.sh (cold drop_caches variant) and bench-ram.sh (peak RSS). It:

• generates reproducible trees (reusing scripts/demo/gen.py),
• runs a matrix of binaries × io-threads × walk-threads × env-variants over declarative workload profiles,
• cold (default, via the sudoers tee /proc/sys/vm/drop_caches) or --warm, interleaved across rounds with within-round shuffling,
• reports median/mean/min/max wall + user + sys (+ --rss peak RSS).

Profiles live in a dict (realistic, mixed, bigfile, many, big, git) — future needs add a key, not a script. scripts/demo/* is untouched (that's the GIF, not benchmarking).

scripts/bench.py                                              # realistic, cold, 5 rounds
scripts/bench.py -p mixed --bin base=/tmp/old --bin new=./oans   # A/B two builds
scripts/bench.py -p git --walk-threads 4,8,16,32             # thread sweep
scripts/bench.py -p many --io-threads 2,4,8 --rss            # + peak RSS

DUPEREMOVE_WALK_THREADS — benchmark hook + a recorded dead-end

Overrides only the walker-thread count (decoupled from the csum/dedupe pools) so bench.py can sweep it. Measured conclusion: decoupling or doubling the walkers does nothing — they only feed the single __scan_file consumer, so past ~4 they're never the bottleneck; the csum pool (--io-threads) is. Cold sweeps (io-threads pinned at 8):

walkers	~/git median	many median
4	14.35 s	8.33 s
8	14.48 s	8.01 s
16	14.52 s	7.88 s
32	14.58 s	8.01 s

Verified real (not a no-op): the same harness's io-threads control swept 1→2→8 = 38.0/20.7/7.9 s, DUPEREMOVE_WALK_THREADS=N produced exactly N live walker threads, and each run stored 250k extents. Default (unset) behavior is unchanged.

CLAUDE.md dead-ends

Also records that separating the walk from hashing into two sequential phases isn't worth it — cold-benchmarked as byte-identical output, neutral-to-~3%-worse (it throws away the walk/hash pipeline overlap).

Test

scripts/verify.sh passes: build (warnings = failure), 94 unit+integration tests, valgrind scan+dedupe+replay smoke.

🤖 Generated with Claude Code