Conversation
…th widths EXPERIMENTAL successor of mh built on Mul128Fold64 (MUM): unrolled per-length small-key terminal actions (data in both product operands - quadratic products kill pairwise correlations), length folded into the seed, 128-byte 8-chain bulk fetch window with distinct chain secrets, native dual-lane 128-bit form, two-multiply finalizer keeping both widening-product halves. Secrets are sqrt fractions of the first 16 primes. Three measured iterations (v1 175 -> v2 177 -> v3 188/188 both widths, the only clean native 128 measured on our rig); fastest mixed-length latency <= 16B and fastest bulk throughput in the library. Also merges SMHASHER3.md into mbo/hash/README.md (Principles -> Performance -> Quality) with same-run benchmark tables; end-game decisions (mh retirement, defaults, name, streaming) in TODO.md.
- Rename mh2 -> mumbo (MUM + mbo), drop the EXPERIMENTAL label; v4: the length moves from the seed into the finalizer's product operands (streaming-enabling; equally protective) and the 128-bit lane seeds derive from distinct secret pairs so no lane equals the 64-bit hash. - Streaming for mumbo: eager 128-byte block consumption + rolling last-16 window; chunked == one-shot (typed test). - mumbo becomes DefaultHashAlgorithm AND Default128HashAlgorithm. - New //mbo/hash:hash_extra_cc (mbo/hash/hash_extra.h): the NOTICE-bearing transcriptions (rapidhash MIT, xxh64/xxh3 BSD-2) now require explicit linkage; the default hash_cc is notice-free. - Drop mh entirely (never released): hash_mh.h deleted, all references scrubbed from code, tests, README, and CHANGELOG (which now presents mumbo directly). - README: tables updated (mumbo v4 numbers), mumbo design-iteration section replaces the mh failure analysis. v4 SMHasher3 re-verification is running; the doc marks it in flight.
…ble break Every build/test/benchmark job now gates on both lint jobs (benchmark previously gated on nothing, tests only on pre-commit), so a trunk failure stops the matrix early. Also fixes the markdownlint MD056 that trunk caught: the quality table's trailing paragraph had lost its blank line and was absorbed as a table row.
'mumbo jumbo': jumbo::Algorithm (inherits the complete mumbo surface) fronts the native 128-bit form and becomes Default128HashAlgorithm; both structs are complete so either plugs in anywhere. Docs batch: root README hash section (mumbo/jumbo defaults, extras via hash_extra.h, hash_extra_cc target bullet), third-party tables point compliance at hash_extra_cc (hash_cc is notice-free), NOTICE preamble updated accordingly, TODO records the resolved hash_extra decision and the remaining jumbo-streaming idea, hash README tables renamed (jumbo 128 rows) with v4 64-bit re-verification recorded (PASS 188).
… extend perf tables to all algorithms fnv1a (the std::hash family baseline, e.g. MSVC), murmur3, siphash, and the legacy simple join the latency/throughput tables; overview table maps each algorithm to its target (hash.h default vs hash_extra_cc opt-in), NOTICE obligation, seeding, and streaming. Same-rig SMHasher3 batteries for fnv1a/murmur3/siphash are running; their quality rows land when complete.
…88), siphash (PASS) The std::hash baseline quantified: fnv1a passes 7 of 186 tests; murmur3 fails the modern Seed* cluster broadly; siphash is clean as a PRF must be. Overview cells and quality table complete for all algorithms; methodology lists the built-in registrations used.
helly25
changed the title
Fab-Cat
approved these changes
Jul 5, 2026
…live in the README data sections)
…n the named rule is linked)
helly25
added a commit
that referenced
this pull request
Jul 5, 2026
* docs(hash): CI cross-platform performance tables (x86_64 gcc vs arm64 clang) Latency and throughput from PR #235's CI benchmark artifacts (ubuntu-latest / macos-26, mean of 3 repetitions), with the noise and constant-folding caveats stated. The mumbo/rapidhash near-tie and jumbo's 128-bit lead reproduce on both architectures. * docs(hash): drop trailing colons from perf headings (markdownlint MD026)
helly25
added a commit
that referenced
this pull request
Jul 6, 2026
- Root README: CombineHashes / Hash64To32 / Streamer and the per-algorithm functions were accidentally nested under the hash_mangle_cc heading; they are hash.h API and moved back. The mangle flag documentation now uses the one-per-bullet 'Custom Bazel flag' convention of the config section. - CHANGELOG: the mangle bullet presents the released design directly instead of describing the replacement of the MBO_HASH_MANGLE define and the __DATE__/__TIME__ bucketing - both existed only within the unreleased 0.13.0 cycle (same treatment as the never-released mh in #235). - mbo/hash/README: the Offerings pointer distinguishes the mangle section (semantics, design) from the Configuration section (flags).
helly25
added a commit
that referenced
this pull request
Jul 7, 2026
…e-version rotation (#238) * feat(hash): flag-driven build-seed mangle - hash_mangle.h / :hash_mangle_cc split - GetHash / MangledHasher move out of hash.h into the new public entry mbo/hash/hash_mangle.h (//mbo/hash:hash_mangle_cc): hash.h / :hash_cc users are never exposed to (or rebuilt for) seed variation. - The mangle constant is generated into a header by folding the module's own version (native.module_version() - every release rotates it by construction, for free) with two custom flags: --//mbo/hash:mangle_seed (any printable-ASCII string) and --//mbo/hash:mangle_seed_buckets (default 8; 0 disables making GetHash == GetHash64, 1 pins one stable constant across releases and seeds). - Neither the version nor raw seed strings reach C++ action keys: both fold to a bucket inside the generation rule, so caches see at most N + 1 header variants. Replaces the per-TU __DATE__/__TIME__ bucketing and the MBO_HASH_MANGLE define (and their ODR risk). - internal/hash_mangle_seed.h.in is the non-Bazel fallback; a diff_test pins it byte-identical to the default-flag generation (regen command in its header). CI additionally runs hash_mangle_test with buckets=0. - hash_mangle_test (replaces hash_no_mangle_test): seed-independent XOR properties, exact values via a transparent DummyAlgorithm, constexpr static_asserts, enabled<=>differs sweeps, and a typed sweep over algo::AllAlgorithms (extras included - tests do not ship, so the NOTICE-driven target split does not constrain test deps). - READMEs: Offerings and Configuration sections, design rationale (constexpr rules out ASLR; buckets bound the churn), mumbo lineage (simple -> mh -> mh2 -> mumbo). * style(hash): migrate hash_test.cc to STYLE_CPP.md test conventions - EXPECT_THAT/ASSERT_THAT with Eq/Ne/Lt/Gt/Le matchers replace all 77 comparison macros; container size checks use SizeIs. - All 25 bare TESTs become TEST_F with per-suite empty fixtures. - NOLINTNEXTLINE(readability-function-cognitive-complexity) lines carry reasons; the StreamingMatchesOneShot rng NOLINT now sits on the line clang-tidy reports on. - mbo/types/tstring.h: fix the stale StringHash comment (GetHash64). * docs(hash): release-consistency pass - Root README: CombineHashes / Hash64To32 / Streamer and the per-algorithm functions were accidentally nested under the hash_mangle_cc heading; they are hash.h API and moved back. The mangle flag documentation now uses the one-per-bullet 'Custom Bazel flag' convention of the config section. - CHANGELOG: the mangle bullet presents the released design directly instead of describing the replacement of the MBO_HASH_MANGLE define and the __DATE__/__TIME__ bucketing - both existed only within the unreleased 0.13.0 cycle (same treatment as the never-released mh in #235). - mbo/hash/README: the Offerings pointer distinguishes the mangle section (semantics, design) from the Configuration section (flags). * style(hash): fix NOLINT placement in hash_mumbo.h - BulkBlock: the *-constant-array-index suppression sat on the for-line; as NOLINTNEXTLINE it now covers the chain[i] / kSecret[4 + i] line clang-tidy actually reports on. - StreamUpdate: bracket the pointer/array-walking body with NOLINTBEGIN/END(cppcoreguidelines-pro-bounds-constant-array-index). * docs(hash): Abseil interop section Compose, do not compete: containers take DefaultHasher outright for byte keys; structured types inject mbo values (Streamer -> Finalize) into AbslHashValue combining (Hash128/tstring precedent); absl::HashOf folded into mbo values imports per-process randomization (in-process only); a full absl::Hash replacement via a custom combine/combine_contiguous state is possible by protocol design but a deliberate separate project.
helly25
added a commit
that referenced
this pull request
Jul 7, 2026
…all (#239) * feat(hash): flag-driven build-seed mangle - hash_mangle.h / :hash_mangle_cc split - GetHash / MangledHasher move out of hash.h into the new public entry mbo/hash/hash_mangle.h (//mbo/hash:hash_mangle_cc): hash.h / :hash_cc users are never exposed to (or rebuilt for) seed variation. - The mangle constant is generated into a header by folding the module's own version (native.module_version() - every release rotates it by construction, for free) with two custom flags: --//mbo/hash:mangle_seed (any printable-ASCII string) and --//mbo/hash:mangle_seed_buckets (default 8; 0 disables making GetHash == GetHash64, 1 pins one stable constant across releases and seeds). - Neither the version nor raw seed strings reach C++ action keys: both fold to a bucket inside the generation rule, so caches see at most N + 1 header variants. Replaces the per-TU __DATE__/__TIME__ bucketing and the MBO_HASH_MANGLE define (and their ODR risk). - internal/hash_mangle_seed.h.in is the non-Bazel fallback; a diff_test pins it byte-identical to the default-flag generation (regen command in its header). CI additionally runs hash_mangle_test with buckets=0. - hash_mangle_test (replaces hash_no_mangle_test): seed-independent XOR properties, exact values via a transparent DummyAlgorithm, constexpr static_asserts, enabled<=>differs sweeps, and a typed sweep over algo::AllAlgorithms (extras included - tests do not ship, so the NOTICE-driven target split does not constrain test deps). - READMEs: Offerings and Configuration sections, design rationale (constexpr rules out ASLR; buckets bound the churn), mumbo lineage (simple -> mh -> mh2 -> mumbo). * style(hash): migrate hash_test.cc to STYLE_CPP.md test conventions - EXPECT_THAT/ASSERT_THAT with Eq/Ne/Lt/Gt/Le matchers replace all 77 comparison macros; container size checks use SizeIs. - All 25 bare TESTs become TEST_F with per-suite empty fixtures. - NOLINTNEXTLINE(readability-function-cognitive-complexity) lines carry reasons; the StreamingMatchesOneShot rng NOLINT now sits on the line clang-tidy reports on. - mbo/types/tstring.h: fix the stale StringHash comment (GetHash64). * docs(hash): release-consistency pass - Root README: CombineHashes / Hash64To32 / Streamer and the per-algorithm functions were accidentally nested under the hash_mangle_cc heading; they are hash.h API and moved back. The mangle flag documentation now uses the one-per-bullet 'Custom Bazel flag' convention of the config section. - CHANGELOG: the mangle bullet presents the released design directly instead of describing the replacement of the MBO_HASH_MANGLE define and the __DATE__/__TIME__ bucketing - both existed only within the unreleased 0.13.0 cycle (same treatment as the never-released mh in #235). - mbo/hash/README: the Offerings pointer distinguishes the mangle section (semantics, design) from the Configuration section (flags). * style(hash): fix NOLINT placement in hash_mumbo.h - BulkBlock: the *-constant-array-index suppression sat on the for-line; as NOLINTNEXTLINE it now covers the chain[i] / kSecret[4 + i] line clang-tidy actually reports on. - StreamUpdate: bracket the pointer/array-walking body with NOLINTBEGIN/END(cppcoreguidelines-pro-bounds-constant-array-index). * docs(hash): Abseil interop section Compose, do not compete: containers take DefaultHasher outright for byte keys; structured types inject mbo values (Streamer -> Finalize) into AbslHashValue combining (Hash128/tstring precedent); absl::HashOf folded into mbo values imports per-process randomization (in-process only); a full absl::Hash replacement via a custom combine/combine_contiguous state is possible by protocol design but a deliberate separate project. * bench(hash): boundary-resolved small-key sizes + regenerated throughput tables The throughput benchmark stepped by x4 (1,4,16,64,...), jumping straight over the small-string-optimization band. Add tier/SSO boundary lengths - 7/8 (end of the fully-unrolled <=8 path), 15/16 (<=16 path; 15 = libstdc++ SSO cap), 22 (libc++ SSO cap), 32 (proposed cap), 63/64 (short-chain edge) - so the small-key cliffs are visible. Regenerate the README 64- and 128-bit throughput tables and the latency table from one consistent run over all algorithms at these sizes, and rewrite the reading: the boundary data shows mumbo trails rapidhash by ~0.3-0.7 ns on inline-string keys (2.45 ns at the 15B cap, 2.49 ns at 22B), the deliberate cost of the clean-both-widths finalizer; the deficit does not carry into the dependency-bound latency case. * perf(hash): mumbo small-key LoadSmall - if-ladder instead of jump-table switch Diagnosis: mumbo's small-key deficit vs rapidhash (~0.3-0.7 ns) is dominated by load/dispatch, not the finalizer - a one-multiply probe still trailed rapidhash by 0.34 ns, and the two-multiply finalizer is what earns the clean 188/188 in both widths. The 10-case LoadSmall switch compiled to a jump table (table load + indirect branch); replacing it with an if-ladder that gates the common len>=4 range first lets 9..16 (the bulk of hashed keys and the SSO range) resolve on the first compare. Values are byte-identical to the switch at every length 0..16 (verified by construction: each branch reproduces the exact loads/expressions; >16 does not use LoadSmall), so no re-verification is needed and streaming/one-shot parity is unchanged. Min-of-30 throughput, rapidhash-normalized: ~0.1-0.25 ns faster at 1/7/15 B, flat at 8/16, no regression. * bench(hash): measurement infrastructure - dev module, report tool, fast/full modes Dev-only measurement tooling for //mbo/hash:hash_benchmark, isolated as its own Bazel module (mbo/hash/measurements/, listed in .bazelignore) so none of it - including future plotting deps - enters the helly25_mbo module or any offering. - hash_benchmark.cc: two size modes. FAST (default, CI, README tables) is a dense boundary/SSO-straddling set; FULL (MBO_HASH_BENCHMARK_FULL=1) is a ~3x denser slow-exponential sweep for the ns-vs-length curve. - hash_benchmark_report.py (stdlib only): run / store / tables / plot. Reads google/benchmark's native JSON (no transcode). Headline aggregate is the MEAN OF THE 3 FASTEST OF 9 reps - the low tail approximates the uncontended cost, and averaging the best few rejects the single-sample fluke a pure min keeps; best_cv reports the spread of those 3 (not the contention-laden 9). Stores full provenance: date, git SHA + dirty + branch (warns when not a clean main tree, since a squash-merge discards a dev SHA), host/CPU/OS, load, cpu_scaling. Precautions: -c opt, random interleaving, warmup, 9 reps (google/benchmark compare.py U-test minimum). Transposed markdown tables (length per row) and a dependency-free SVG curve. - README perf tables regenerated (best-3-of-9, transposed); design doc in mbo/hash/measurements/README.md covers methodology, provenance, storage (canonical JSON committed as text; raw gzipped, authoritative-only), and the measure-after-merge workflow. - CI benchmark job: main-only, gated on mbo/hash changes (dorny/paths-filter), already continue-on-error, now with the same precautions. Numbers are provisional (dev branch); the authoritative dataset + committed canonical JSON land post-merge on a clean main checkout. * bench(hash): timestamped outputs, pre-aligned tables, SMHasher3 mode - Every written file is prefixed YYYYMMDD_HHMMSS_ (one stamp per invocation), so runs never overwrite and the filename records when it was produced. - The tables command now emits vertically aligned markdown directly (shared padding helper; first column right-aligned, rest left) - no reformat needed. - New 'smhasher' subcommand: runs the SMHasher3 battery over --algos (default 'all', explicitly including the legacy simple), parses pass/fail + failing families, stores JSON with provenance plus each run's full log. It drives a built SMHasher3 (--smhasher3); the in-house mumbo/jumbo/simple need a patched SMHasher3 that registers them - building that harness is the remaining piece (design doc updated). * bench(hash): smhasher mode parses score + failing families The stored entry now carries the SMHasher3 score (passed/total, e.g. 181/188) and the list of failing test/family names (e.g. Avalanche, Sparse [20/3], SeedBIC [8]), printed per algorithm and stored in the results JSON; the full per-run log (the complete why) is saved alongside. Tolerant parser: verdict + optional (X / Y) score, failing lines minus the overall-verdict line. * refactor(hash)!: rename legacy 'simple' hash to 'dumbo', drop deprecated API The legacy in-house hash was misnamed 'simple' - it is not simpler than mumbo, it is the -umbo family's weak member (64-bit, unseeded, does not pass SMHasher3), kept only for comparison. Rename mbo::hash::simple -> mbo::hash::dumbo (mumbo/jumbo/dumbo) so the name signals in-house + legacy. BREAKING (pre-1.0, so no alias kept): the mbo::hash::simple namespace and the already-deprecated simple::GetHash wrapper are removed; migrate to mbo::hash::GetHash64 (mumbo) for real hashing, or mbo::hash::dumbo::GetHash64 for the legacy hash. - hash_simple.h -> hash_dumbo.h (namespace + GetDumboHash), deprecated GetHash dropped; added a TODO to actually improve dumbo (it is competitive for tiny 1-8B keys, where a better finalizer could pay off - SMHasher3-gated). - Updated hash.h/BUILD, the test descriptor (DumboHash) + tests, the measurement tool (dumbo-64), both READMEs, and the CHANGELOG. Build + all hash tests green; no live mbo::hash::simple references remain. * bench(hash): reproducible SMHasher3 build harness (build_smhasher3.sh) Encodes the README 'Methodology (reproduction)' as a script: clone SMHasher3 at the pinned commit (6ab4343), apply the two documented fixes (missing <cstdlib> in lib/AEStest.cpp; replace -march=native with a portable arch), and build with gcc in a container. Third-party algorithms verify immediately via 'report.py smhasher --smhasher3 <path>'; the in-house mumbo/jumbo/dumbo registration source (SMHasher3 hashes/ plugin) is the remaining piece, to be run on merged main for an authoritative result. Design doc updated. * docs(hash): measurements - fix stray 'simple' and 'minimum' aggregate wording Two leftover references to the pre-rename name / pre-best-k aggregate: smhasher section said 'legacy simple' (now dumbo), and the script docstring still described the aggregate as the minimum (now the mean of the k fastest). * feat(hash): give legacy dumbo (weak) seed support dumbo ignored its seed, so it failed SMHasher3's seed sanity by construction and could not be seed-tested. Fold the seed into the initial state (one XOR, carried through the existing multiply chain) - negligible cost, and it now reacts to the seed. dumbo::GetHash64 gains an optional seed; the descriptor is kSeeded. Still a weak legacy hash, not a strong mixer. (First step on the improve-dumbo TODO; unblocks its SMHasher3 seed tests.) Verified: //mbo/hash:hash_test + hash_mangle_test green (const==runtime, seed-avalanche reacts); READMEs + CHANGELOG updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The in-house mumbo/jumbo hash family (MUM + mbo; "mumbo jumbo"): built on the widening 64x64->128 multiply, developed in four measured iterations (each: benchmark + full SMHasher3 batteries, documented in
mbo/hash/README.md), and now the library default in all forms.Headlines
mumbo(64) joins rapidhash as the only clean 64-bit passes;jumbo(128) is the only clean 128-bit result we have measured (xxh3-128: 162, murmur3: 123).jumbois the fastest 128-bit hash from 16 bytes up (1.7-2.4x xxh3-128).DefaultHashAlgorithm = mumbo::Algorithm,Default128HashAlgorithm = jumbo::Algorithm. Both structs are complete (each delegates the other width), so either plugs in anywhere.//mbo/hash:hash_extra_cc(hash_extra.h): the NOTICE-bearing transcriptions (rapidhash MIT, xxh64/xxh3 BSD-2) now require explicit linkage - the defaulthash_ccis notice-free. NOTICE/README third-party tables updated accordingly.Design (hash_mumbo.h)
Unrolled per-length small-key terminal actions with the key in BOTH product operands (quadratic products kill pairwise correlations); 17-127B sequential MUM chain with overlapping end loads (no tail loops anywhere); >= 128B eight chains over a 128-byte fetch window with distinct secrets; finalizer keeps both widening-product halves with the length in the product operands; 128-bit form runs two lanes with distinct secret banks and swapped operand roles over a shared 4-chain bulk tier. Secrets: sqrt fractions of the first 16 primes (= SHA-512/SHA-384 IVs).
The data story (mbo/hash/README.md)
hash.hdefault vshash_extra_ccopt-in), NOTICE obligation, seeding, streaming, SMHasher3 verdict.fnv1a, the family behind manystd::hashimplementations (fastest possible 1-byte hash; 10-70x slower than mumbo beyond).Also
trunkandpre-commit.HasStreamingconcept extension).CI benchmark results (run 28753339340; informational - shared runners are noisy)
Values are
ubuntu-latest(x86_64, gcc, 4 vCPU) /macos-26(arm64, Apple clang) mean of 3 repetitions, ns/op.Mixed-length latency (serialized chain):
* gcc constant-folding artifacts on the fixed-size lanes; treat with suspicion.
64-bit one-shot:
128-bit one-shot:
Cross-platform reading: the mumbo/rapidhash near-tie holds on both architectures (rapidhash leads x86_64-gcc bulk, they tie on arm64); jumbo is the fastest 128-bit hash from 256 bytes up on BOTH platforms; the xxh3 mid-size dip and the fnv1a/siphash/simple profiles reproduce everywhere.