Skip to content

mbo/hash/measurements: one-shot authoritative runner + parallel SMHasher batteries - #242

Merged
helly25 merged 2 commits into
mainfrom
hash-measurements-runner
Jul 9, 2026
Merged

mbo/hash/measurements: one-shot authoritative runner + parallel SMHasher batteries#242
helly25 merged 2 commits into
mainfrom
hash-measurements-runner

Conversation

@helly25

@helly25 helly25 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Dev-only tooling under the .bazelignore'd mbo/hash/measurements module - no shipped code changes. Makes the authoritative measurement run a single command and cuts the SMHasher3 wall-clock.

run_measurements.py (new)

One command, run from a clean main checkout, that:

  1. runs the full performance sweep (solo - it is sub-ns and SMHasher3 would contend for CPU),
  2. renders the ns-vs-length chart,
  3. runs the SMHasher3 battery (parallel),
  4. promotes the canonical JSON + SVG to stable committed names and prints exactly what to commit.
mbo/hash/measurements/run_measurements.py --jobs 4          # everything
mbo/hash/measurements/run_measurements.py --algos mumbo,jumbo,dumbo --jobs 1
mbo/hash/measurements/run_measurements.py --skip-smhasher   # perf + chart only

hash_benchmark_report.py smhasher gains --jobs

The per-algorithm batteries are independent, and their pass/fail verdicts are load-independent (only SMHasher3's own Speed sub-test would be affected by contention, and that number is unused - performance comes from our own benchmark). So --jobs N runs several concurrently, trading cores for wall-clock (--algos all drops from ~2 h to well under 1 h).

--smhasher3 is now a command prefix (shell-split), so it works both as a native binary path and as a docker run ... ./build/SMHasher3 wrapper. That's needed on macOS, where the container-built Linux binary can't run natively; the runner constructs the wrapper automatically (image matches build_smhasher3.sh).

Notes

  • The committed dataset is only tagged authoritative from a clean main tree; the runner warns otherwise.
  • data/ stays .gitignored; the runner prints the git add -f commands for the raw + SMHasher artifacts, and the plain git add for the canonical JSON + chart.
  • Measured artifact sizes are repo-friendly: raw ~316 KB gzip, canonical ~100 KB, SVG ~16 KB, each SMHasher log ~20 KB gzip.

helly25 added 2 commits July 9, 2026 20:45
…asher

Adds `run_measurements.py`: a single command (run from clean `main`) that does
the full perf sweep, renders the ns-vs-length chart, and runs the SMHasher3
battery, then prints exactly what to commit. Perf runs first and alone (it is
sub-ns and SMHasher3 would contend for CPU); the batteries follow.

`hash_benchmark_report.py smhasher` gains `--jobs`: the per-algorithm batteries
are independent and their pass/fail verdicts are load-independent (only
SMHasher3's own Speed sub-test would be, and that number is unused - perf is our
own benchmark), so several run concurrently, trading cores for wall-clock
(~2 h -> well under 1 h). `--smhasher3` is now treated as a command *prefix*
(shell-split), so it works as a native binary path AND as a `docker run ...`
wrapper - needed on macOS, where the container-built Linux binary is invoked
via Docker (the runner constructs that wrapper automatically).

Dev-only tooling under the .bazelignore'd measurements module; no shipped code
changes.
@helly25
helly25 requested a review from Fab-Cat July 9, 2026 20:26
@helly25
helly25 merged commit 81ddc2a into main Jul 9, 2026
7 checks passed
@helly25
helly25 deleted the hash-measurements-runner branch July 9, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants