Skip to content

zeroed_fe_vec: reinterpret via from_raw_parts, not Vec transmute#797

Merged
MauroToscano merged 1 commit into
perf/tracegen-cpu-optimizationsfrom
review/pr786-calloc-from-raw-parts
Jul 8, 2026
Merged

zeroed_fe_vec: reinterpret via from_raw_parts, not Vec transmute#797
MauroToscano merged 1 commit into
perf/tracegen-cpu-optimizationsfrom
review/pr786-calloc-from-raw-parts

Conversation

@MauroToscano

Copy link
Copy Markdown
Contributor

Small hardening of the zeroed_fe_vec helper added in the opt commit (calloc-backed zeroed trace buffers). No behavior change — pure implementation swap.

Why

The helper reinterpreted its calloc'd buffer with:

unsafe { core::mem::transmute::<Vec<u64>, Vec<FE>>(zeros) }

Transmuting one Vec to another relies on Vec's field layout being identical across element types. Vec is #[repr(Rust)] (unspecified layout, permitted to depend on the element type), so this isn't guaranteed by the language — the std mem::transmute docs call out this exact VecVec pattern as a Bad Idea and recommend from_raw_parts. It's sound on the pinned rustc 1.94.0 (no layout randomization), but e.g. -Zrandomize-layout could permute {ptr, cap} differently for the two types and make it UB.

What

Rebuild the Vec<FE> from (ptr, len, cap) on the same allocation:

let mut zeros = core::mem::ManuallyDrop::new(zeros);
unsafe { Vec::from_raw_parts(zeros.as_mut_ptr() as *mut FE, zeros.len(), zeros.capacity()) }

Same calloc'd buffer, same element size/align (already const-asserted), so the eventual dealloc Layout matches exactly and there's no dependence on Vec's internal layout. The FE-is-u64 / zero-is-all-zero justification is unchanged and stays self-contained in the helper (no coupling to the disk-spill SpillSafe trait — deliberately, since that's a separate, feature-scoped concern).

Testing

Full cargo test -p lambda-vm-prover --lib: 503 passed (only the 5 known missing rust-guest ELF fixtures fail, at file-read). The zeroed_fe_vec_matches_fe_zero guard test and end-to-end prove+verify pass, confirming byte-identical behavior. make lint, clippy --all-targets, fmt --check clean.

zeroed_fe_vec built its Vec<FE> with mem::transmute::<Vec<u64>, Vec<FE>>.
That relies on Vec's field layout being identical across element types, which
the language does not guarantee (Vec is #[repr(Rust)], and the std transmute
docs flag exactly this Vec-to-Vec pattern as a Bad Idea). It happens to work on
the pinned toolchain, but a layout change (e.g. -Zrandomize-layout) could swap
the ptr/cap words and make it UB.

Rebuild the Vec from its raw parts instead: same calloc'd allocation, same
element size/align (already const-asserted), so the dealloc Layout matches and
there is no dependence on Vec's internal layout. Behavior is unchanged — the
zeroed_fe_vec_matches_fe_zero test and full prove+verify still pass.
@MauroToscano MauroToscano merged commit 674bbe7 into perf/tracegen-cpu-optimizations Jul 8, 2026
13 checks passed
@MauroToscano MauroToscano deleted the review/pr786-calloc-from-raw-parts branch July 8, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant