refactor(benchmark): split the comparative harness from the CI one

[rmanibus]

Summary

Stacked on #435 — base is refactor/bench-reporting-in-go, so review that first. GitHub retargets this to main automatically once #435 merges. Per repo convention a stacked PR gets no CI run.

What CI was actually doing

benchmark.yml invoked run.sh local local cloudstic — the third argument filters to cloudstic — on a runner where restic, borg and duplicacy are not installed. The comparison never happened in CI. It was a 615-line cross-tool harness executing 64 lines of cloudstic-specific code.

Why that matters beyond the dead paths

The dead paths are cosmetic. The structural problem is that the two jobs have conflicting requirements:

• A comparison needs a dataset that is fair across four tools, and can only measure what all of them expose.
• Tracking this product over time needs a dataset that stresses this design, and is free to report numbers no other tool has.

Sharing one script meant a change made for cross-tool fairness — adjusting the dataset so borg is not unfairly penalised, say — would silently move the numbers a trend line is built on.

The split

Script	Job
compare.sh (was run.sh)	Cloudstic vs restic/borg/duplicacy. Manual only.
cloudstic.sh (new)	This product on its own terms. What the Benchmark workflow runs.
lib.sh (new)	Measurement mechanics both use.

lib.sh holds run_bench, repo-size tracking and the table — the how. The datasets deliberately stay separate, which follows directly from the argument above: sharing them would reintroduce exactly the coupling the split removes.

cloudstic.sh measures things a comparison cannot, and the first run shows why they are worth having:

Operation	Time · Peak RSS	Repo added
Initial Backup	0.7s · 534.4 MB	344.0 MB
Incremental (1 File Changed)	0.27s · 127.3 MB	12 KB
Incremental (1000 Changed)	0.27s · 144.2 MB	1.3 MB
Deduplicated Backup	0.54s · 343.4 MB	40 KB
Prune	0.13s · 153.1 MB	-320 KB

Copying 340 MB of already-stored data adds 40 KB, and a thousand changed files costs the same wall time as one — both properties the design claims and neither previously measured. The script also reports a stored-to-logical ratio (2.18x on this dataset).

Report changes

The table grows a Repo added column for single-tool runs, and the prose no longer says "each tool over the same dataset" when there is one. The column is deliberately withheld across tools, where it would invite comparing byte counts between different repository formats — that is not the same measurement twice.

Verification

• ./scripts/benchmark/cloudstic.sh — full run, output above
• ./scripts/benchmark/compare.sh local local cloudstic — still works after the lib.sh extraction, output unchanged
• /bin/bash -n on all three scripts under bash 3.2.57
• go test -race ./... passes; golangci-lint run ./... — 0 issues
• npx markdownlint-cli2 '**/*.md' — 0 issues

Trigger

The Benchmark workflow had fired once since April 2nd, so the split was worth roughly nothing on its own. It now also runs on a pull request carrying the benchmark label, matching Memory scaling.

Same label rather than a second one, deliberately: labelling a PR is the act of asking for performance data, and wanting the memory curve but not the throughput numbers is not a real case. synchronize is included so pushing to an already-labelled PR re-measures without another click, and any other label leaves both jobs skipped.

[codecov]

Codecov Report

❌ Patch coverage is 88.46154% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/benchreport/render.go	90.47%	1 Missing and 1 partial ⚠️
internal/benchreport/report.go	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!