mbo/hash: known-answer tests + SMHasher3 include-order fix; verify_digest_test bzl rule - #244
Merged
Conversation
…gest_test bzl rule SMHasher3 plugin: include Platform.h before Hashlib.h (Platform.h defines seed_t and the FLAG_IMPL_*/FLAG_HASH_* enums the REGISTER_HASH macros need; this is the order SMHasher3's own hashes use). Pinned behind a clang-format guard, since SortIncludes alphabetizes "Hashlib.h" first and silently reverts it. Re-verified in a container build: mumbo-64, jumbo-128, dumbo-64 all PASS. Known-answer vectors: hash_test_vectors_gen emits hash_test_vectors.inc as one field-named constexpr struct array (reads like textproto, stays a compile-time header, no runtime parser). hash_test's InHouseVectors re-derives every vector from the live algorithm, and per-algorithm CountKatAlgo(...) >= 20 asserts prove every in-house algorithm is covered with enough cases (all dispatch tiers, seeded and unseeded) instead of asserting an arbitrary total. Integrity in three independent layers: - diff_test: committed .inc vs a freshly generated one (fails on a forgotten regeneration after an algorithm or generator change). - verify_digest_test: the committed hash_test_vectors.inc.sha256 sidecar, an independently verifiable artifact (stock `sha256sum -c` or `digest --check`). - InHouseVectors: the runtime value check above. New reusable rule //mbo/digest:digest.bzl verify_digest_test: given an algorithm and files mapped to a saved digest, either `digests` (a checksum sidecar file, preferred and externally verifiable) or `checksums` (an inline hex digest), it re-checks each file with `//mbo/digest:digest --check`, so a file may only change when its saved digest is updated in the same commit. Self-tested in mbo/digest against a fixture covering both forms. Also corrects the measurements/README "Open items" note (the "SMHasher3 build pin is broken" misdiagnosis): the pin 6ab4343 is correct, and the include-order fix here is what unblocks a committed authoritative smhasher.json.
GCC compiles hash_test with -Werror=comment; a '//' line ending in '\' (the regen command wrapped across lines) is a multi-line-comment warning and failed CI. The generator now emits the command unwrapped (clang-format re-wraps it at a space, '>'-terminated, not a backslash), and its own header comment is fixed the same way. Regenerated hash_test_vectors.inc + .sha256. Verified with clang++ -Wcomment -Werror; macOS clang did not flag it originally.
Fab-Cat
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The SMHasher3 build fix and the known-answer tests for the in-house hashes, in one commit (follows #243).
SMHasher3 include-order fix
smhasher3/mbohash.cppincludedHashlib.hbeforePlatform.h.Platform.hdefinesseed_tand theFLAG_IMPL_*/FLAG_HASH_*enums theREGISTER_HASHmacros rely on, so the reversed order put every$.field/flag out of scope and the reproducible build failed. This was the real cause behind the "build pin is broken" note from #243 (a misdiagnosis - the pin6ab4343is correct); thatmeasurements/README"Open items" note is corrected here.The include order is now pinned behind a
// clang-format offguard, becauseSortIncludesalphabetizesHashlib.hfirst and silently reverts it. Re-verified in a fresh container build (build_smhasher3.sh-> gcc:13): mumbo-64, jumbo-128, dumbo-64 all PASS.Known-answer tests for the in-house mumbo/jumbo and dumbo
hash_test_vectors_genemitshash_test_vectors.incas one field-namedconstexprstruct array (reads like textproto, stays a compile-time header, no runtime parser). Regenerate intentionally on an algorithm change; the values are not stable across library versions.Integrity is enforced in three independent layers:
InHouseVectors(hash_test) re-derives every vector from the live algorithm, and per-algorithmCountKatAlgo(...) >= 20asserts prove every in-house algorithm is covered with enough cases (all dispatch tiers, seeded and unseeded) - not an arbitrary total.hash_test_vectors_up_to_date_test(diff_test) compares the committed.incagainst a freshly generated one, so a forgotten regeneration after an algorithm or generator change fails.hash_test_vectors_digest_test(verify_digest_test, below) checks the committedhash_test_vectors.inc.sha256sidecar - an independently verifiable artifact anyone can re-check with stocksha256sum -c.New rule:
//mbo/digest:digest.bzlverify_digest_testGiven an
algorithmand files mapped to a saved digest - eitherdigests(a checksum sidecar file, preferred and externally verifiable) orchecksums(an inline hex digest in the BUILD file) - it re-checks each file with//mbo/digest:digest --check, so a file may only change when its saved digest is updated in the same commit. Self-tested inmbo/digestagainst a fixture covering both forms.Follow-up
With the plugin building reproducibly again, the authoritative
smhasher.jsoncan be produced byrun_measurements.pyon a cleanmaincheckout after this merges (the last "Open items" entry).