zeroed_fe_vec: reinterpret via from_raw_parts, not Vec transmute#797
Merged
MauroToscano merged 1 commit intoJul 8, 2026
Merged
Conversation
zeroed_fe_vec built its Vec<FE> with mem::transmute::<Vec<u64>, Vec<FE>>. That relies on Vec's field layout being identical across element types, which the language does not guarantee (Vec is #[repr(Rust)], and the std transmute docs flag exactly this Vec-to-Vec pattern as a Bad Idea). It happens to work on the pinned toolchain, but a layout change (e.g. -Zrandomize-layout) could swap the ptr/cap words and make it UB. Rebuild the Vec from its raw parts instead: same calloc'd allocation, same element size/align (already const-asserted), so the dealloc Layout matches and there is no dependence on Vec's internal layout. Behavior is unchanged — the zeroed_fe_vec_matches_fe_zero test and full prove+verify still pass.
674bbe7
into
perf/tracegen-cpu-optimizations
13 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Small hardening of the
zeroed_fe_vechelper added in theoptcommit (calloc-backed zeroed trace buffers). No behavior change — pure implementation swap.Why
The helper reinterpreted its calloc'd buffer with:
Transmuting one
Vecto another relies onVec's field layout being identical across element types.Vecis#[repr(Rust)](unspecified layout, permitted to depend on the element type), so this isn't guaranteed by the language — the stdmem::transmutedocs call out this exactVec→Vecpattern as a Bad Idea and recommendfrom_raw_parts. It's sound on the pinnedrustc 1.94.0(no layout randomization), but e.g.-Zrandomize-layoutcould permute{ptr, cap}differently for the two types and make it UB.What
Rebuild the
Vec<FE>from(ptr, len, cap)on the same allocation:Same calloc'd buffer, same element size/align (already
const-asserted), so the eventual deallocLayoutmatches exactly and there's no dependence onVec's internal layout. TheFE-is-u64/ zero-is-all-zero justification is unchanged and stays self-contained in the helper (no coupling to the disk-spillSpillSafetrait — deliberately, since that's a separate, feature-scoped concern).Testing
Full
cargo test -p lambda-vm-prover --lib: 503 passed (only the 5 known missing rust-guest ELF fixtures fail, at file-read). Thezeroed_fe_vec_matches_fe_zeroguard test and end-to-end prove+verify pass, confirming byte-identical behavior.make lint,clippy --all-targets,fmt --checkclean.