feat(hash): SipHash, rapidhash V3, differential test vs libxxhash, latency benchmark - #219
Merged
Conversation
…tency benchmark Medium tier of mbo/hash/TODO.md: - SipHash (hash_siphash.h): constexpr-safe SipHash-c-d (canonical 2-4 via GetHash64, 1-3 via GetHash64Sip13) with the reference 128-bit key API; verified against reference vectors (empty-input value matches the SipHash paper's table). The keyed, hash-flooding-resistant option the library lacked; Algorithm derives the key from the seed as (seed, Fmix64(seed)), documented as requiring a secret seed for adversarial protection. - rapidhash V3 (hash_rapidhash.h): constexpr-safe transcription of the reference FAST variant (wyhash family; small-key latency champion, SMHasher3-clean); verified against vectors generated by compiling the official header across every dispatch branch. hash_internal gains Mult128 (full 64x64->128, both halves). - Differential test (hash_differential_test.cc): our xxh64/xxh3 vs the actual reference library over 40k+ randomized (input, seed, length) cases incl. block boundaries and 100KB buffers - bit-for-bit agreement, proving the scalar transcription matches even the reference's SIMD path. Reference xxHash 0.8.3 pulled as a test-only http_archive (the BCR module's BUILD predates Bazel 9's rule-autoload removal). - Mixed-length latency benchmark (BmHash64Latency): unpredictable key sizes with a serialized hash->index dependency chain. Instantly instructive: the ranking flips vs hot-loop throughput (rapidhash 11ns < xxh3 12.2 < mh 16.6 at 0..16B mixes). TODO.md: medium tier cleared except the one-off SMHasher3 run; XXH3-128 added as a future item; MD5/SHA recorded as a non-goal (use BoringSSL; our fast file-identity answer is XXH3).
…ion over crypto-lib dependency BoringSSL is live-at-head, unversioned, and has a history of non-reproducible archives - an unverifiable supply chain for exactly the code that most needs verifying. Digests are spec-frozen pure functions; if interop is ever needed, transcribe + pin against NIST vectors in-repo.
…erately not 'crypto')
The 32-bit counterpart of Hash128To64: all 64 bits contribute. Plain truncation would be sound for the strong algorithms (full-avalanche finalizers), but the fold is the correct default for every algorithm - FNV-1a's low bits are biased and XOR-folding is that algorithm's official shrinking recommendation.
Adds the HasGetHash32 concept, Hasher<Algo>::GetHash32, and the top-level GetHash32<Algo>(data, seed). Algorithms may provide a native 32-bit variant (e.g. a future canonical XXH32 / FNV-1a-32 / murmur3-x86); everything else synthesizes the XOR-fold of the 64-bit hash (Hash64To32) - the per-algorithm correct approach, selected automatically.
…ty BUILD file Fixes four design problems with the differential test's reference archive: - The BUILD file is a real file (third_party/xxhash/xxhash.BUILD.bazel), not an inline build_file_content string. third_party/ holds per-dependency directories for external BUILD files and patches; the files are named <dep>.BUILD.bazel so they never load as helly25_mbo packages. (A nested MODULE.bazel boundary was tried first: Bazel 9 does not treat it as a package-traversal boundary - the BUILD file loaded as a main-repo package.) - The library is testonly. - Visibility is restricted to //mbo/hash. - The http_archive moved to bazelmod/dev.MODULE.bazel with dev_dependency = True: consumers of helly25_mbo never fetch it. The differential test is tagged manual so consumer wildcards never analyze a target whose dependency does not exist for them; our CI runs it explicitly.
Fab-Cat
approved these changes
Jul 4, 2026
helly25
disabled auto-merge
July 4, 2026 14:22
helly25
added a commit
that referenced
this pull request
Jul 4, 2026
# Conflicts: # CHANGELOG.md # mbo/hash/hash_differential_test.cc # mbo/hash/hash_test.cc
helly25
added a commit
that referenced
this pull request
Jul 4, 2026
# Conflicts: # mbo/hash/TODO.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Medium tier of
mbo/hash/TODO.md, in one batch.SipHash — the security story (
siphash::Algorithm)Constexpr-safe SipHash-c-d: canonical SipHash-2-4 (
GetHash64(data, k0, k1)) and SipHash-1-3 (GetHash64Sip13), reference 128-bit-key API. Verified against reference vectors (the empty-input value0x726fdb47dd0e0e31matches the SipHash paper's test-vector table). TheAlgorithmplug-in derives the key from the 64-bit seed as(seed, Fmix64(seed))— documented: adversarial DoS protection requires a secret seed.rapidhash V3 — the small-key latency champion (
rapidhash::Algorithm)Constexpr-safe transcription of the reference FAST variant (wyhash family, SMHasher3-clean, MIT). Vectors generated by compiling the official
rapidhash.hacross every dispatch branch (0/1–3/4–7/8–16/17–112 ladder/113/224/225/448/1024/3000 × seeds).hash_internal::Mult128added (full 64×64→128, both halves).Differential test vs the real xxHash library
hash_differential_test: our xxh64/xxh3 vs libxxhash 0.8.3 over 40k+ randomized (input, seed, length) cases plus 100 KB buffers — bit-for-bit agreement, including against the reference's SIMD path. Far stronger than fixed vectors. (Pulled as a test-onlyhttp_archive; the BCR module's BUILD predates Bazel 9's rule-autoload removal.)Mixed-length latency benchmark
BmHash64Latency<Algo>/{16,64,1024}: keys of unpredictable random length, each hash result selecting the next key (serialized chain) — measuring what hash-table workloads actually pay. First results (Apple clang) flip the throughput ranking:rapidhash's small-key-latency reputation confirmed; siphash pays the documented DoS-resistance premium.
Also