Skip to content

scripts: add bench-dedupe.py, a larger-than-RAM dedupe benchmark harness - #121

Merged
martinus merged 1 commit into
masterfrom
feat/bench-dedupe
Jul 24, 2026
Merged

scripts: add bench-dedupe.py, a larger-than-RAM dedupe benchmark harness#121
martinus merged 1 commit into
masterfrom
feat/bench-dedupe

Conversation

@martinus

Copy link
Copy Markdown
Owner

What

Commits the dedupe-phase benchmark harness that keeps getting reconstructed by hand for each release. scripts/bench.py deliberately only measures the scan/hash phase (-rq, non-destructive); it can't measure dedupe — dedupe mutates the tree (each round needs a fresh unshared copy) and its signature cost only shows up when the working set is larger than RAM. scripts/bench-dedupe.py fills that gap and produces the numbers in docs/benchmarks.md's Larger-than-RAM dedupe section.

What it does

  • Builds N --reflink=never copies of a --source tree (dedupe reclaims the later copies against copy0).
  • Times two phases per binary — hash (-rq, in-memory) and dedupe (-dr --hashfile) — cold (drop_caches) inside a MemoryMax-capped systemd-run --user --scope, interleaved in randomized order, restoring the copies + dropping the hashfile between dedupe rounds.
  • Reports wall/user/sys/RSS/disk-read as median/mean/min/max (+ hashfile size).
  • --verify confirms byte-identical sharing across binaries via btrfs filesystem du -s.
  • --baseline build:REF builds a comparison binary from a git ref in a cached worktree (build:897a222 = the fork's pure-upstream base, the only valid duperemove baseline); or pass a path, or omit it to bench oans alone.
  • Reuses bench.py's drop_caches / reflink-fs helpers rather than duplicating them.

Canonical A/B:

scripts/bench-dedupe.py --baseline build:897a222 --source ~/git/linux --cap 4G --rounds 10 --verify

Verified

  • py_compile + --help clean.
  • End-to-end smoke (2 copies of a small tree, both phases, --verify, capped scope, --clean): tables render, hashfile sizes captured, verify reports IDENTICAL sharing, cleanup works.
  • The build:897a222 baseline path: adds the worktree, builds it, runs, and --clean removes the worktree.

Dev tooling + docs only — no source changes. docs/benchmarks.md "Reproduce it" and CLAUDE.md now point at the tool.

🤖 Generated with Claude Code

bench.py only benchmarks the scan/hash phase (-rq, non-destructive). It can't
measure dedupe: dedupe mutates the tree, so each timed round needs a fresh
unshared copy, and its signature cost only appears when the working set is
larger than RAM. That harness kept getting reconstructed by hand for every
release; this commits it.

bench-dedupe.py builds N non-reflinked copies of a source tree, times the
hash-only and hash+dedupe phases cold (drop_caches) inside a MemoryMax-capped
`systemd-run --user --scope`, restores the copies + drops the hashfile between
dedupe rounds, and reports wall/user/sys/RSS/disk-read as median/mean/min/max.
--verify confirms byte-identical sharing across binaries via
`btrfs filesystem du -s`. A --baseline of build:REF builds a comparison binary
from a git ref in a cached worktree (e.g. build:897a222 = the fork's upstream
base). It reuses bench.py's drop_caches/reflink helpers.

This produces docs/benchmarks.md's "Larger-than-RAM dedupe" numbers; the doc's
"Reproduce it" and CLAUDE.md now point at it (the one sanctioned exception to
"no new bench-*", since bench.py structurally can't measure dedupe).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@martinus
martinus force-pushed the feat/bench-dedupe branch from 4eea867 to 4011f53 Compare July 24, 2026 14:28
@martinus
martinus merged commit dfb94dc into master Jul 24, 2026
6 checks passed
@martinus
martinus deleted the feat/bench-dedupe branch July 24, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant