feat(#169): MemorySanitizer via an instrumented libc++ (Linux CI) - #432
Merged
Conversation
The -hasheq matcher and --summary=hash dedup grouping cover the practical hash-verification workflow. A one-pass verified/failed tally needs the per-entry verdict threaded thread-safely through the reduction feed (parallel walk / tsan); parked with that note until a concrete need appears.
MSan catches reads of uninitialized memory that asan does not. The one real prerequisite is an instrumented C++ standard library: MSan false-positives on anything it did not instrument, and while every other dep builds from source under bazel (so -fsanitize=memory reaches it), libc++ ships prebuilt inside the hermetic LLVM. - tools/build_msan_libcxx.sh builds libc++/libc++abi/libunwind from the LLVM source release matching bazelmod/llvm.MODULE.bazel's llvm_version, with -DLLVM_USE_SANITIZER=MemoryWithOrigins and the hermetic clang as compiler, into .msan-libcxx/ (gitignored). It stamps the prefix with version+script hash so a cached tree is reused, and skips cleanly on non-Linux. - .bazelrc --config=msan mirrors asan/tsan and swaps the runtime in via -nostdinc++ -isystem .../include/c++/v1 and -nostdlib++ -L... -Wl,-rpath,... - CI gains one ubuntu `msan` cell (never macOS: MSan has no macOS support) with the instrumented libc++ cached on the LLVM version + script hash. It starts continue-on-error while first findings are triaged - the same introduction path the clang-tidy cell took - then becomes a hard gate. Also folds the ratified archive control surface into the docs so nothing is left open before that work starts: --archive[=none|roots|all] + -z/-z+/-z- with find -> none and the xff family -> roots, dual container identity, --archive-depth=1, sniff-gating, and the pre-implementation "not yet implemented" guard. Rewrites the stale smart-case entry (it shipped as --case=smart/-s, not a boolean), and resolves the remaining design.md TBDs (PCRE2 spelling is --regextype=PCRE2; the VFS seam is built, remote backends stay post-v1).
helly25
enabled auto-merge (squash)
August 9, 2026 22:35
# Conflicts: # TODO.md
The first msan CI run failed at cmake configure: the runtimes CMakeLists add_subdirectory()s llvm/utils/llvm-lit for its check targets, which the partial source extraction did not include. Turn the test machinery off (-DLLVM_INCLUDE_TESTS=OFF; we only want the libraries) and extract llvm/utils as well so the path exists either way.
Second configure/compile failure from the hand-picked extraction: libcxx's from_chars_floating_point.h includes shared/fp_bits.h, which lives under libc/, just as the earlier failure needed llvm/utils/llvm-lit. Picking directories is a losing game - each attempt only surfaces the next missing one - so extract the whole source release. It costs seconds, the tree is discarded afterwards, and what CI caches is the INSTALLED runtimes, not the sources.
The done job asserts that EVERY declared workflow job is listed in its needs - a guard against adding a job that silently is not gated - and adding msan without wiring it tripped exactly that check. msan belongs in needs like the rest: the gate step only inspects the needs KEY SET, never the results, and a continue-on-error job cannot fail the run. So listing it restores the completeness invariant while a red msan still cannot block a merge, which is the whole point of introducing it report-only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enables MemorySanitizer - reads of uninitialized memory, which asan does not catch - as a Linux-only CI cell.
The prerequisite, solved
MSan false-positives on any code it did not instrument. Every dep we link builds from source under bazel (abseil, re2, googletest, mbo, pcre2), so
--config=msan's-fsanitize=memoryreaches them for free. The exception is the C++ standard library, which ships prebuilt and uninstrumented inside the hermetic LLVM.tools/build_msan_libcxx.shbuildslibc++/libc++abi/libunwindfrom the LLVM source release matchingbazelmod/llvm.MODULE.bazel'sllvm_version(so runtimes can never drift from the compiler), with-DLLVM_USE_SANITIZER=MemoryWithOriginsand the hermetic clang, into.msan-libcxx/. It stamps the prefix with version + script hash so a cached tree is reused, and skips cleanly on non-Linux.Wiring
.bazelrc--config=msanmirrors asan/tsan, swapping the runtime in with-nostdinc++ -isystem .../include/c++/v1and-nostdlib++ -L... -Wl,-rpath,... -lc++ -lc++abi.msanCI cell (never macOS - MSan has no macOS support), with the instrumented libc++ cached on the LLVM version + script hash.continue-on-errorwhile the first findings are triaged, the same introduction path theclang-tidycell took, then becomes a hard gate indone'sneeds. This PR's msan cell result is therefore informational; the real validation is what it reports.Also: closing every open design question
Per the "no open questions before 0.2.0 / archive" sweep:
--archive[=none|roots|all]+-z-/-z/-z+,find->noneand the xff family ->roots, dual container identity,--archive-depth=1, sniff-gating inallmode, raw-compressed files as one-member archives. Supersedes the older "always recurse, not roots-only" framing.--archivewith a distinct "not yet implemented" usage error, never a silent no-op and never confused with the lean build's "not built in".--case=smart/-s/-s+(a value of the three-state--case), not a--smart-caseboolean.--regextype=PCRE2; the VFS seam is built and remote backends stay post-v1.A repo-wide grep for
[DISCUSS]/Open:/TBD/undecidedacross TODO.md, docs/, AGENTS.md and README.md now returns nothing.bazel test //xff/... --config=clanggreen (89 pass); pre-commit clean; workflow YAML validated.