Suppress UBSan enum check on SPIR-V mask bit-combining operators#5
Merged
bghgary merged 1 commit intoBabylonJS:mainfrom Apr 24, 2026
Merged
Conversation
Bit-combining operations on SPIR-V mask enums routinely produce values that are not among the individual enumerators (e.g. ~MaskX or A | B). Under -fsanitize=enum this is diagnosed as UB. Scope the suppression to just these operator functions; the pattern is intentional. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bghgary
added a commit
to bghgary/BabylonNative
that referenced
this pull request
Apr 24, 2026
Now that BabylonJS/glslang#5 (Suppress UBSan enum check on SPIR-V mask bit-combining operators) has merged, point FetchContent at the merged-on-main SHA instead of the bghgary fork branch used for testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bghgary
added a commit
to bghgary/BabylonNative
that referenced
this pull request
Apr 24, 2026
Now that BabylonJS/glslang#5 (Suppress UBSan enum check on SPIR-V mask bit-combining operators) has merged, point FetchContent at the merged-on-main SHA instead of the bghgary fork branch used for testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bghgary
added a commit
to BabylonJS/BabylonNative
that referenced
this pull request
Apr 25, 2026
## Context Our sanitized macOS/Linux jobs enable `-fsanitize=address,undefined[,vptr]`, so the UB checks are active — but **UBSan is recoverable by default**. On a hit it prints a diagnostic to stderr and keeps running. The process doesn't abort, the test doesn't fail, CI stays green, and the diagnostic is buried in test output nobody reads. Adding `-fno-sanitize-recover=all` is what converts each UBSan hit into a `SIGABRT` → test failure. We were missing it. ## Change Three commits: 1. **Make UBSan findings fatal in sanitized builds.** Add `-fno-sanitize-recover=all` to the Clang sanitizer `add_compile_options` line in `CMakeLists.txt`, and set `UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1:symbolize=1` and `ASAN_OPTIONS=abort_on_error=1` in `build-macos.yml` / `build-linux.yml` as runtime belt-and-suspenders. (MSVC only supports ASan, which already aborts on error.) 2. **Bump glslang to merged PR #5 SHA** — picks up [BabylonJS/glslang#5](BabylonJS/glslang#5) (suppress UBSan enum check on SPIR-V mask bit-combining operators). 3. **Bump bgfx.cmake to merged #115 (UBSan fix)** — picks up [BabylonJS/bgfx.cmake#115](BabylonJS/bgfx.cmake#115), which bumps the bgfx submodule to the cherry-pick of [bkaradzic/bgfx#3688](bkaradzic/bgfx#3688) (default-init the SortKey ints/bools so we don't load undefined values into bitfields) via [BabylonJS/bgfx#60](BabylonJS/bgfx#60). The dependency bumps in commits 2 and 3 fix every UBSan finding currently being printed (and ignored) by the sanitized jobs on `master`, so this PR should be green on first run. ## Out of scope - **metal-cpp's intentional "nil-messaging" pattern.** Apple documents sending messages to `nil` as well-defined in Obj-C, but C++ UBSan doesn't know that. Not hit on the current pin; if a future metal-cpp bump trips it, the standard fix is a scoped `-fno-sanitize=null` on metal-cpp only. - `-fsanitize=nullability` (Apple `_Nonnull` attribute checks, separate from `null`) — worth considering as a follow-up once this lands and the baseline is clean. - Windows ASan already aborts on error; no change needed there. [Created by Copilot on behalf of @bghgary] --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
operator|/operator&/operator^/operator~overloads inSPIRV/spirv.hppexist specifically to bit-combine mask-enum values. By construction the results are usually NOT one of the individually-declared enumerators —A | B,~A, etc. produce composite bit patterns that no enumerator matches on its own.Under
-fsanitize=enum, creating an enum value whose bit pattern isn't a valid enumerator is reported as UB, e.g. from BabylonNative's macOS sanitized CI:These diagnostics were previously printed to stderr and ignored. With BabylonNative now running with
-fno-sanitize-recover=all, they become hard aborts and block the build.Fix
Wrap only the mask bit-combining operator block (
spirv.hpp:2685-2724) with#pragma clang attribute push(__attribute__((no_sanitize("enum"))), apply_to = function)/pop. The pattern is intentional and well-defined at the bit level; the suppression is scoped to exactly the functions where the design requires it, so the rest of the codebase continues to benefit from UBSan's enum check.Conditioned on
__clang__; other compilers (GCC, MSVC) are unaffected.[Created by Copilot on behalf of @bghgary]