scripts: add bench-dedupe.py, a larger-than-RAM dedupe benchmark harness - #121
Merged
Conversation
bench.py only benchmarks the scan/hash phase (-rq, non-destructive). It can't measure dedupe: dedupe mutates the tree, so each timed round needs a fresh unshared copy, and its signature cost only appears when the working set is larger than RAM. That harness kept getting reconstructed by hand for every release; this commits it. bench-dedupe.py builds N non-reflinked copies of a source tree, times the hash-only and hash+dedupe phases cold (drop_caches) inside a MemoryMax-capped `systemd-run --user --scope`, restores the copies + drops the hashfile between dedupe rounds, and reports wall/user/sys/RSS/disk-read as median/mean/min/max. --verify confirms byte-identical sharing across binaries via `btrfs filesystem du -s`. A --baseline of build:REF builds a comparison binary from a git ref in a cached worktree (e.g. build:897a222 = the fork's upstream base). It reuses bench.py's drop_caches/reflink helpers. This produces docs/benchmarks.md's "Larger-than-RAM dedupe" numbers; the doc's "Reproduce it" and CLAUDE.md now point at it (the one sanctioned exception to "no new bench-*", since bench.py structurally can't measure dedupe). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
martinus
force-pushed
the
feat/bench-dedupe
branch
from
July 24, 2026 14:28
4eea867 to
4011f53
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Commits the dedupe-phase benchmark harness that keeps getting reconstructed by hand for each release.
scripts/bench.pydeliberately only measures the scan/hash phase (-rq, non-destructive); it can't measure dedupe — dedupe mutates the tree (each round needs a fresh unshared copy) and its signature cost only shows up when the working set is larger than RAM.scripts/bench-dedupe.pyfills that gap and produces the numbers indocs/benchmarks.md's Larger-than-RAM dedupe section.What it does
--reflink=nevercopies of a--sourcetree (dedupe reclaims the later copies againstcopy0).hash(-rq, in-memory) anddedupe(-dr --hashfile) — cold (drop_caches) inside aMemoryMax-cappedsystemd-run --user --scope, interleaved in randomized order, restoring the copies + dropping the hashfile between dedupe rounds.--verifyconfirms byte-identical sharing across binaries viabtrfs filesystem du -s.--baseline build:REFbuilds a comparison binary from a git ref in a cached worktree (build:897a222= the fork's pure-upstream base, the only valid duperemove baseline); or pass a path, or omit it to bench oans alone.bench.py'sdrop_caches/ reflink-fs helpers rather than duplicating them.Canonical A/B:
scripts/bench-dedupe.py --baseline build:897a222 --source ~/git/linux --cap 4G --rounds 10 --verifyVerified
py_compile+--helpclean.--verify, capped scope,--clean): tables render, hashfile sizes captured, verify reportsIDENTICAL sharing, cleanup works.build:897a222baseline path: adds the worktree, builds it, runs, and--cleanremoves the worktree.Dev tooling + docs only — no source changes.
docs/benchmarks.md"Reproduce it" andCLAUDE.mdnow point at the tool.🤖 Generated with Claude Code