Skip to content

feat(hash): XXH3-64, container functor adapters, CombineHashes, TODO roadmap - #215

Merged
helly25 merged 5 commits into
mainfrom
hash/roadmap-batch1
Jul 3, 2026
Merged

feat(hash): XXH3-64, container functor adapters, CombineHashes, TODO roadmap#215
helly25 merged 5 commits into
mainfrom
hash/roadmap-batch1

Conversation

@helly25

@helly25 helly25 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Batch 1 from the state-of-the-art gap analysis (mbo/hash/TODO.md).

XXH3 (64-bit) — xxh3::Algorithm

Constexpr-safe scalar implementation of the modern xxHash generation, transcribed from the reference implementation v0.8.2 (secret byte-for-byte, all dispatch classes: 0 / 1–3 / 4–8 / 9–16 / 17–128 / 129–240 / long with 1024-byte blocks + scramble + seeded custom-secret path). Canonical XXH3_64bits[_withSeed] values pinned by 34 reference vectors covering every class boundary (× seeds 0/5381).

New hash_internal::Mul128Fold64 (constexpr __uint128_t where available, portable 32-bit schoolbook otherwise) — the xxh3/wyhash-family core, so a future wyhash-class algorithm is now unblocked.

Container functor adapters

Hasher<Algo> is now a transparent functor (operator() + is_transparent):

absl::flat_hash_map<std::string, V, mbo::hash::DefaultHasher, std::equal_to<>> map;
map.find(std::string_view(...));  // heterogeneous, no temporary std::string

CombineHashes

mbo::hash::CombineHashes(uint64_t, uint64_t) — order-dependent, Fmix64-finalized golden-ratio combine.

Roadmap

mbo/hash/TODO.md records the gap analysis (easy/medium/hard/non-goals) in-repo; done items removed per its policy.

Algorithm comparison (CI benchmark, this branch)

All cells: throughput in GiB/s (mean of 3); bracketed factor is relative to xxh64 in the same row (xxh64 = 1.00x). Bold marks the fastest algorithm per size.

ubuntu-latest (x86_64, gcc)

len simple mh fnv1a xxh64 xxh3 murmur3
1 B 0.10 (0.42x) 0.33 (1.44x) 1.38 (6.00x) 0.23 (1.00x) 0.42 (1.85x) 0.20 (0.87x)
4 B 0.48 (0.48x) 1.31 (1.31x) 1.88 (1.89x) 1.00 (1.00x) 1.91 (1.92x) 0.80 (0.80x)
16 B 0.51 (0.17x) 5.26 (1.76x) 1.63 (0.55x) 2.99 (1.00x) 8.32 (2.78x) 2.45 (0.82x)
64 B 0.52 (0.12x) 4.94 (1.14x) 1.04 (0.24x) 4.35 (1.00x) 14.05 (3.23x) 4.45 (1.02x)
256 B 0.52 (0.07x) 8.76 (1.26x) 0.79 (0.11x) 6.96 (1.00x) 1.89 (0.27x) 5.07 (0.73x)
1 KB 0.52 (0.06x) 10.96 (1.34x) 0.76 (0.09x) 8.19 (1.00x) 3.93 (0.48x) 5.14 (0.63x)
4 KB 0.52 (0.06x) 11.57 (1.35x) 0.75 (0.09x) 8.57 (1.00x) 5.36 (0.63x) 5.29 (0.62x)

macos-26 (arm64, Apple clang)

len simple mh fnv1a xxh64 xxh3 murmur3
1 B 0.80 (2.24x) 0.43 (1.20x) 1.28 (3.57x) 0.36 (1.00x) 0.39 (1.09x) 0.24 (0.68x)
4 B 3.15 (2.10x) 1.65 (1.10x) 1.76 (1.18x) 1.50 (1.00x) 1.68 (1.12x) 0.90 (0.60x)
16 B 4.88 (1.19x) 6.03 (1.47x) 2.23 (0.54x) 4.11 (1.00x) 6.50 (1.58x) 2.76 (0.67x)
64 B 4.16 (0.51x) 6.85 (0.84x) 1.28 (0.16x) 8.17 (1.00x) 10.63 (1.30x) 5.00 (0.61x)
256 B 2.78 (0.22x) 13.76 (1.09x) 0.83 (0.07x) 12.64 (1.00x) 6.46 (0.51x) 5.19 (0.41x)
1 KB 1.97 (0.16x) 16.21 (1.28x) 0.70 (0.06x) 12.64 (1.00x) 13.46 (1.07x) 4.50 (0.36x)
4 KB 1.88 (0.15x) 14.89 (1.19x) 0.69 (0.06x) 12.47 (1.00x) 18.57 (1.49x) 4.30 (0.34x)

Reading

  • xxh3 owns the 4–64 B band on both platforms (up to 3.2x xxh64 at 64 B on gcc) and the 4 KB top end on macOS (18.6 GiB/s).
  • mh owns the 256 B–1 KB middle band everywhere, and stays the best all-rounder (never below 0.84x of the winner outside 1–4 B).
  • The scalar xxh3 dips at 256 B–1 KB (most visibly on gcc): that is where the reference implementation switches to its SIMD long path, whose fixed 8-accumulator setup only amortizes at KB+ in scalar form. Expected behavior for a constexpr-safe scalar transcription, not a bug — the values stay canonical.
  • fnv1a still wins the 1-byte corner; simple the 4 B corner on macOS.

All typed framework tests cover xxh3 automatically (avalanche 0.45–0.55 at every tier incl. the >240 B long path, collisions, constexpr==runtime, width detection).

helly25 added 5 commits July 3, 2026 21:27
The two-seed fallback was only as strong as the algorithm's seed handling:
seed-affine hashes collide both lanes together and seed-ignoring ones
produced h1 == h2. The second pass now skips the first up-to-8 bytes and
injects them (with kSeedFlip) into its seed, so both lanes cover every
input byte yet hash different data -- the lanes decorrelate even for
algorithms with weak or ignored seed handling, at unchanged cost.

Alternatives considered: a plain +8 shift leaves lane 2 blind to the first
8 bytes (structural collision class for any algorithm); a half/half split
is 2x faster but each output bit would depend on only half the input.
…roadmap

Batch from the state-of-the-art gap analysis (see mbo/hash/TODO.md):

- XXH3 (64-bit): constexpr-safe scalar implementation of the modern xxHash
  generation, transcribed from the reference implementation v0.8.2 (192-byte
  kSecret byte-for-byte; all dispatch classes 0/1-3/4-8/9-16/17-128/129-240/
  long incl. blocks, scramble, and the seeded custom-secret path). Canonical
  XXH3_64bits[_withSeed] values pinned by 34 reference vectors across every
  class boundary. Registered as xxh3::Algorithm; benchmark: fastest at 64B
  (15.6 GiB/s) and 4KB (21.5 GiB/s) on Apple clang.
- hash_internal::Mul128Fold64: constexpr 64x64->128 fold (uint128 where
  available, portable 32-bit schoolbook otherwise) - also unblocks a future
  wyhash-class algorithm.
- Hasher<Algo> is now a transparent functor (operator() + is_transparent):
  drops directly into absl/std hash containers with heterogeneous
  string_view lookup.
- mbo::hash::CombineHashes(uint64_t, uint64_t): order-dependent, well-mixed
  combine.
- mbo/hash/TODO.md: the prioritized roadmap from the gap analysis (easy /
  medium / hard / non-goals), maintained in-repo.
# Conflicts:
#	CHANGELOG.md
#	README.md
@helly25
helly25 requested a review from Fab-Cat July 3, 2026 22:11
@helly25
helly25 enabled auto-merge (squash) July 3, 2026 22:43
@helly25
helly25 merged commit e11fa2b into main Jul 3, 2026
40 checks passed
@helly25
helly25 deleted the hash/roadmap-batch1 branch July 3, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants