Skip to content

mbo/hash: mumbo/jumbo - in-house MUM hash family takes the defaults; hash_extra_cc split; full same-rig data#235

Merged
helly25 merged 11 commits into
mainfrom
hash/mh2
Jul 5, 2026
Merged

mbo/hash: mumbo/jumbo - in-house MUM hash family takes the defaults; hash_extra_cc split; full same-rig data#235
helly25 merged 11 commits into
mainfrom
hash/mh2

Conversation

@helly25

@helly25 helly25 commented Jul 5, 2026

Copy link
Copy Markdown
Owner

The in-house mumbo/jumbo hash family (MUM + mbo; "mumbo jumbo"): built on the widening 64x64->128 multiply, developed in four measured iterations (each: benchmark + full SMHasher3 batteries, documented in mbo/hash/README.md), and now the library default in all forms.

Headlines

  • SMHasher3: PASS 188/188 in BOTH widths (same-rig): mumbo (64) joins rapidhash as the only clean 64-bit passes; jumbo (128) is the only clean 128-bit result we have measured (xxh3-128: 162, murmur3: 123).
  • Performance: trades blows with rapidhash across the 64-bit board (leads bulk at 46.1 GiB/s and ties small-key latency); jumbo is the fastest 128-bit hash from 16 bytes up (1.7-2.4x xxh3-128).
  • Streaming: v4 moved the length from the seed into the finalizer's product operands (equally protective, known only at finalize) - enabling exact chunked==one-shot streaming (eager 128-byte blocks + rolling last-16 window).
  • Defaults: DefaultHashAlgorithm = mumbo::Algorithm, Default128HashAlgorithm = jumbo::Algorithm. Both structs are complete (each delegates the other width), so either plugs in anywhere.
  • //mbo/hash:hash_extra_cc (hash_extra.h): the NOTICE-bearing transcriptions (rapidhash MIT, xxh64/xxh3 BSD-2) now require explicit linkage - the default hash_cc is notice-free. NOTICE/README third-party tables updated accordingly.
  • mh dropped entirely (never released): file deleted, all references scrubbed; the CHANGELOG presents mumbo directly.

Design (hash_mumbo.h)

Unrolled per-length small-key terminal actions with the key in BOTH product operands (quadratic products kill pairwise correlations); 17-127B sequential MUM chain with overlapping end loads (no tail loops anywhere); >= 128B eight chains over a 128-byte fetch window with distinct secrets; finalizer keeps both widening-product halves with the length in the product operands; 128-bit form runs two lanes with distinct secret banks and swapped operand roles over a shared 4-chain bulk tier. Secrets: sqrt fractions of the first 16 primes (= SHA-512/SHA-384 IVs).

The data story (mbo/hash/README.md)

  • New Algorithm overview table: widths, availability (hash.h default vs hash_extra_cc opt-in), NOTICE obligation, seeding, streaming, SMHasher3 verdict.
  • Performance tables extended to ALL algorithms - including fnv1a, the family behind many std::hash implementations (fastest possible 1-byte hash; 10-70x slower than mumbo beyond).
  • Same-rig SMHasher3 for everything: mumbo 188 / jumbo 188 / rapidhash 188 / siphash 186-of-186 clean; xxh64 181, xxh3 166/162, murmur3 123, fnv1a 7 of 186.
  • The measured iteration history (v1 175 -> v2 177 -> v3 188/188 -> v4 re-verified 188/188 with streaming enabled) with root-cause analysis per step.

Also

  • CI: all build/test/benchmark jobs now gate on BOTH trunk and pre-commit.
  • TODO records the resolved hash_extra decision and the one open idea (128-bit streaming needs a HasStreaming concept extension).

CI benchmark results (run 28753339340; informational - shared runners are noisy)

Values are ubuntu-latest (x86_64, gcc, 4 vCPU) / macos-26 (arm64, Apple clang) mean of 3 repetitions, ns/op.

Mixed-length latency (serialized chain):

max len mumbo rapidhash xxh3 xxh64 murmur3 siphash fnv1a simple
16 7.5 / 9.8 8.1 / 10.9 7.7 / 12.0 12.6 / 14.1 12.4 / 16.1 15.4 / 22.2 12.5 / 13.2 20.4 / 9.8
64 9.3 / 11.2 10.1 / 11.4 3.6* / 12.6 25.4 / 21.4 15.7 / 20.1 22.2 / 34.5 0.9* / 38.1 69.9 / 26.7
1024 41.4 / 29.2 18.9 / 27.4 111.5 / 44.2 120.1 / 74.6 105.1 / 78.4 216 / 282 581 / 570 837 / 374

* gcc constant-folding artifacts on the fixed-size lanes; treat with suspicion.

64-bit one-shot:

size mumbo rapidhash xxh3 xxh64
16B 3.1 / 3.0 3.4 / 2.3 1.7 / 2.6 5.4 / 3.5
256B 23.2 / 12.5 13.6 / 9.5 121.8 / 40.4 33.6 / 19.5
4KiB 275 / 103.0 183 / 102.6 749 / 210 445 / 308

128-bit one-shot:

size mumbo (jumbo) xxh3 murmur3
16B 9.3 / 4.4 3.2 / 3.7 9.0 / 8.9
256B 19.6 / 13.3 128.5 / 43.8 47.9 / 53.6
4KiB 184 / 127 683 / 214 721 / 863

Cross-platform reading: the mumbo/rapidhash near-tie holds on both architectures (rapidhash leads x86_64-gcc bulk, they tie on arm64); jumbo is the fastest 128-bit hash from 256 bytes up on BOTH platforms; the xxh3 mid-size dip and the fnv1a/siphash/simple profiles reproduce everywhere.

helly25 added 9 commits July 5, 2026 19:17
…th widths

EXPERIMENTAL successor of mh built on Mul128Fold64 (MUM): unrolled
per-length small-key terminal actions (data in both product operands -
quadratic products kill pairwise correlations), length folded into the
seed, 128-byte 8-chain bulk fetch window with distinct chain secrets,
native dual-lane 128-bit form, two-multiply finalizer keeping both
widening-product halves. Secrets are sqrt fractions of the first 16
primes. Three measured iterations (v1 175 -> v2 177 -> v3 188/188 both
widths, the only clean native 128 measured on our rig); fastest
mixed-length latency <= 16B and fastest bulk throughput in the library.

Also merges SMHASHER3.md into mbo/hash/README.md (Principles ->
Performance -> Quality) with same-run benchmark tables; end-game
decisions (mh retirement, defaults, name, streaming) in TODO.md.
- Rename mh2 -> mumbo (MUM + mbo), drop the EXPERIMENTAL label; v4: the
  length moves from the seed into the finalizer's product operands
  (streaming-enabling; equally protective) and the 128-bit lane seeds
  derive from distinct secret pairs so no lane equals the 64-bit hash.
- Streaming for mumbo: eager 128-byte block consumption + rolling
  last-16 window; chunked == one-shot (typed test).
- mumbo becomes DefaultHashAlgorithm AND Default128HashAlgorithm.
- New //mbo/hash:hash_extra_cc (mbo/hash/hash_extra.h): the
  NOTICE-bearing transcriptions (rapidhash MIT, xxh64/xxh3 BSD-2) now
  require explicit linkage; the default hash_cc is notice-free.
- Drop mh entirely (never released): hash_mh.h deleted, all references
  scrubbed from code, tests, README, and CHANGELOG (which now presents
  mumbo directly).
- README: tables updated (mumbo v4 numbers), mumbo design-iteration
  section replaces the mh failure analysis. v4 SMHasher3 re-verification
  is running; the doc marks it in flight.
…ble break

Every build/test/benchmark job now gates on both lint jobs (benchmark
previously gated on nothing, tests only on pre-commit), so a trunk
failure stops the matrix early. Also fixes the markdownlint MD056 that
trunk caught: the quality table's trailing paragraph had lost its blank
line and was absorbed as a table row.
'mumbo jumbo': jumbo::Algorithm (inherits the complete mumbo surface)
fronts the native 128-bit form and becomes Default128HashAlgorithm;
both structs are complete so either plugs in anywhere. Docs batch:
root README hash section (mumbo/jumbo defaults, extras via
hash_extra.h, hash_extra_cc target bullet), third-party tables point
compliance at hash_extra_cc (hash_cc is notice-free), NOTICE preamble
updated accordingly, TODO records the resolved hash_extra decision and
the remaining jumbo-streaming idea, hash README tables renamed
(jumbo 128 rows) with v4 64-bit re-verification recorded (PASS 188).
… extend perf tables to all algorithms

fnv1a (the std::hash family baseline, e.g. MSVC), murmur3, siphash, and
the legacy simple join the latency/throughput tables; overview table
maps each algorithm to its target (hash.h default vs hash_extra_cc
opt-in), NOTICE obligation, seeding, and streaming. Same-rig SMHasher3
batteries for fnv1a/murmur3/siphash are running; their quality rows
land when complete.
…88), siphash (PASS)

The std::hash baseline quantified: fnv1a passes 7 of 186 tests; murmur3
fails the modern Seed* cluster broadly; siphash is clean as a PRF must
be. Overview cells and quality table complete for all algorithms;
methodology lists the built-in registrations used.
@helly25 helly25 changed the title mbo/hash: mh2 - widening-multiply redesign, SMHasher3 188/188 in both widths mbo/hash: mumbo/jumbo - in-house MUM hash family takes the defaults; hash_extra_cc split; full same-rig data Jul 5, 2026
@helly25 helly25 requested a review from Fab-Cat July 5, 2026 20:25
@helly25 helly25 merged commit 813fbcd into main Jul 5, 2026
8 checks passed
@helly25 helly25 deleted the hash/mh2 branch July 5, 2026 21:00
helly25 added a commit that referenced this pull request Jul 5, 2026
* docs(hash): CI cross-platform performance tables (x86_64 gcc vs arm64 clang)

Latency and throughput from PR #235's CI benchmark artifacts
(ubuntu-latest / macos-26, mean of 3 repetitions), with the noise and
constant-folding caveats stated. The mumbo/rapidhash near-tie and
jumbo's 128-bit lead reproduce on both architectures.

* docs(hash): drop trailing colons from perf headings (markdownlint MD026)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants