Skip to content

mbo/hash: redesign dumbo-64 into a compact single-lane MUM hash (40 -> 188/188, 2-3x faster) - #241

Merged
helly25 merged 4 commits into
mainfrom
hash-dumbo-redesign
Jul 9, 2026
Merged

mbo/hash: redesign dumbo-64 into a compact single-lane MUM hash (40 -> 188/188, 2-3x faster)#241
helly25 merged 4 commits into
mainfrom
hash-dumbo-redesign

Conversation

@helly25

@helly25 helly25 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Rebuilds the legacy dumbo hash the measured way, addressing both problems the review flagged: silly constants and a slow multi-op core. Lands clean at SMHasher3 188/188.

Result

  • SMHasher3: FAIL 40/188 -> PASS 188/188 (all three in-house hashes now clean).
  • ~2-3x faster for >=8 B; the fastest hash in the suite for 7-8 B; strong (not merely reactive) avalanche.

What changed

  • Constants: silly (6571, 17, 193, 104729) -> nothing-up-my-sleeve (golden ratio + sqrt-prime fractions).
  • Core: the multi-op per-4-byte step -> one Mul128Fold64(word ^ kWord, state ^ kState) MUM step over 8-byte words. Both operands state/data dependent -> product is quadratic; a constant multiplier is linear and fails collisions/distribution no matter the constant.
  • Finalizer: a two-multiply widening avalanche that keeps BOTH halves of the first product and mixes them, with the seed injected into a product operand (dumbo is single-lane, so a seed folded only at init reaches the finalizer weakly mixed for zero-data keys -> SeedZeroes; injecting it at finalize makes the output quadratic in the seed).
  • Length folded at the end (folding it at init let a single low bit cancel a length delta and collide 1-bit with all-zero keys).
  • Loads: shared endian-safe Load64/LoadTail -- no raw memcpy, no signed-char bug, const-eval == runtime on every target (incl. cross/big-endian).
  • Deliberately minimal next to mumbo: one accumulator, one 8-byte word/step, no small-key switch, no lanes, no streaming, no 128-bit form.

Measured design iterations (in README)

legacy 40/188 -> v1 (non-widening, 132/188, exposes the constant-multiplier linearity) -> v2 (MUM core, 186/188, residual SeedZeroes) -> v3 (seed-at-finalize, 188/188).

Tradeoffs (documented)

  • Values changed (dumbo is not stable across versions); plugin verification re-baked.
  • Serial single accumulator -> degrades on large keys (far less than legacy: 256 B 68.7 -> 31.8 ns).
  • The two-multiply finalizer is ~0.15-0.35 ns slower on tiny keys than an fmix64 would be (still fastest at 7-8 B).
  • Loop micro-opts (end-pointer ptr < limit, [[likely]]/[[unlikely]]) were measured and rejected: the loop is latency-bound on the serial MUM chain, so the leaner form is off the critical path (clean A/B showed no difference).

Docs

  • README quality tables (dumbo now PASS 188/188), a "dumbo: the measured design iterations" narrative, perf columns refreshed on the same rig (other algorithms' verified run preserved; cross-platform dumbo row is CI-sourced and refreshes on the next main CI run).

hash_test green (avalanche + seed-avalanche now asserted strong, distribution, structured-key distinctness, const-eval == runtime).

helly25 added 3 commits July 8, 2026 19:43
The legacy dumbo used silly constants (multiply by 6571, add 17/193, a
104729 tail) and a multi-op per-4-byte step: barely diffused (SMHasher3
40/188) and slow. Rebuilt the measured way, keeping it deliberately minimal
next to mumbo (one accumulator, one 8-byte word/step, no small-key switch,
no lanes, no streaming, no 128-bit, stock Fmix64 finalizer):

- nothing-up-my-sleeve constants (golden ratio, sqrt-prime fractions)
- the MUM primitive Mul128Fold64(word^kWord, state^kState): both operands
  state/data dependent, so the product is quadratic (a constant multiplier is
  linear and fails collisions/distribution no matter the constant)
- length folded at the end (folding it at init let a single low bit cancel a
  length delta and collide 1-bit with all-zero keys)
- shared endian-safe Load64/LoadTail: no raw memcpy, no signed-char bug,
  const-eval == runtime on every target (incl. cross/big-endian)

SMHasher3 FAIL 186/188 (from 40); ~2-3x faster than legacy for >=8 B and the
fastest hash in the suite for <=16 B. The two residual failures are both in
SeedZeroes (weak seeding: for zero-data keys the seed enters one operand
linearly) - closing them (stronger finalizer/seed mixing) is the tracked next
step, deferred to keep dumbo minimal. Values changed (dumbo is not stable
across versions); plugin verification re-baked; README quality tables + a
dumbo design-iterations narrative added.
Follow-on to the dumbo redesign (which reached 186/188): the two residual
failures were both SMHasher3 SeedZeroes. Root cause: dumbo is single-lane, so
for zero-data keys a seed folded only at init rides one weakly-mixing chain
and reaches the finalizer nearly linear (mumbo avoids this via 8 seeded bulk
lanes). Fix: replace the fmix64 finalizer with a two-multiply widening
avalanche that keeps BOTH halves of the first product and mixes them, and
inject the seed into a product operand directly -> the output is quadratic in
the seed even for zero-data keys, clearing both SeedZeroes windows.

Result: clean PASS 188/188, and dumbo now has strong (not merely reactive)
avalanche (hash_test kStrongAvalanche flipped to true). Cost: the two widening
multiplies are ~0.15-0.35 ns slower on tiny keys than fmix64, still the
fastest hash in the suite at 7-8 B. Plugin verification re-baked; README
(quality tables now PASS, perf column, v3 iteration) + measurements updated.

The loop micro-opts (end-pointer ptr<limit, likely/unlikely) were measured and
rejected: the loop is latency-bound on the serial MUM chain, so the leaner
form is off the critical path (clean A/B showed no difference).
@helly25 helly25 changed the title mbo/hash: redesign dumbo-64 into a compact single-lane MUM hash (40 -> 186/188, 2-3x faster) mbo/hash: redesign dumbo-64 into a compact single-lane MUM hash (40 -> 188/188, 2-3x faster) Jul 8, 2026
@helly25
helly25 requested a review from Fab-Cat July 9, 2026 05:59
@helly25
helly25 enabled auto-merge (squash) July 9, 2026 05:59
@helly25
helly25 merged commit 74feff1 into main Jul 9, 2026
23 checks passed
@helly25
helly25 deleted the hash-dumbo-redesign branch July 9, 2026 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants