Lower transmutes from int to pointer type as gep on null #121282

saethlin · 2024-02-19T00:46:53Z

I thought of this while looking at #121242. See that PR's description for why this lowering is preferable.

The UI test that's being changed here crashes without changing the transmutes into casts. Based on that, this PR should not be merged without a crater build-and-test run.

rustbot · 2024-02-19T00:47:00Z

r? @fmease

rustbot has assigned @fmease.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

saethlin · 2024-02-19T01:01:52Z

@bors try @rust-timer queue

bors · 2024-02-19T01:03:01Z

⌛ Trying commit 903dd09 with merge d073071...

…e, r=<try> Lower transmutes from int to pointer type as gep on null I thought of this while looking at rust-lang#121242 The UI test that's being changed here crashes without changing the transmutes into casts. Based on that, this PR should not be merged without a crater build-and-test run.

bors · 2024-02-19T02:31:31Z

☀️ Try build successful - checks-actions
Build commit: d073071 (d073071d77ce0f93b4fd8cc567a1e2b9e1b22126)

rust-timer · 2024-02-19T03:45:45Z

Finished benchmarking commit (d073071): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.5%	[3.5%, 3.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.1%	[2.1%, 2.2%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.1%	[0.1%, 0.1%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.1%	[0.1%, 0.1%]	1

Bootstrap: 640.336s -> 641.418s (0.17%)
Artifact size: 308.82 MiB -> 308.83 MiB (0.00%)

saethlin · 2024-02-19T03:47:53Z

@craterbot run mode=build-and-test

craterbot · 2024-02-19T03:48:05Z

👌 Experiment pr-121282 created and queued.
🤖 Automatically detected try build d073071
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

saethlin · 2024-02-20T21:15:59Z

@craterbot abort

craterbot · 2024-02-20T21:16:05Z

🗑️ Experiment pr-121282 deleted!

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

saethlin · 2024-02-20T21:19:34Z

@craterbot run mode=build-and-test start=master#61223975d46f794466efa832bc7562b9707ecc46+rustflags=-Copt-level=3 end=try#d073071d77ce0f93b4fd8cc567a1e2b9e1b22126+rustflags=-Copt-level=3

craterbot · 2024-02-20T21:19:43Z

👌 Experiment pr-121282 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

scottmcm · 2024-02-26T18:55:00Z

tests/codegen/transmute-scalar.rs

@@ -49,7 +49,7 @@ pub fn ptr_to_int(p: *mut u16) -> usize {
 }

 // CHECK: define{{.*}}ptr @int_to_ptr([[USIZE]] %i)
-// CHECK: %_0 = inttoptr [[USIZE]] %i to ptr
+// CHECK: %_0 = getelementptr i8, ptr null, i64 %i


I think to make this pass on non-64-bit you need

Suggested change

// CHECK: %_0 = getelementptr i8, ptr null, i64 %i

// CHECK: %_0 = getelementptr i8, ptr null, [[USIZE]] %i

craterbot · 2024-02-28T17:22:44Z

🚧 Experiment pr-121282 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

RalfJung · 2024-03-11T07:06:04Z

library/core/src/intrinsics.rs

+    ///     std::mem::transmute::<usize, *const u8>(address)
+    /// };
+    /// unsafe {
+    ///     assert_eq!(*ptr, 0);


Suggested change

/// assert_eq!(*ptr, 0);

/// assert_eq!(*ptr, 0); // undefined behavior! ⚠️

RalfJung · 2024-03-11T07:06:23Z

library/core/src/intrinsics.rs

+    /// let address = unsafe {
+    ///     std::mem::transmute::<*const u8, usize>(ptr)
+    /// };
+    /// let new_ref = unsafe {


Suggested change

/// let new_ref = unsafe {

/// // undefined behavior! ⚠️

/// let new_ref = unsafe {

RalfJung · 2024-03-11T07:07:09Z

library/core/src/intrinsics.rs

+    /// Distinguishing exactly which operation in this program is invalid touches on unspecified
+    /// aspects of the Rust memory model. If you need to store a pointer as another type then
+    /// recover the original pointer later, you cannot use `transmute` to round-trip through a
+    /// type which is not a pointer, reference, or [`MaybeUninit`][crate::mem::MaybeUninit].


Seems more useful to say which type can be used?

RalfJung · 2024-03-11T07:12:12Z

library/core/src/intrinsics.rs

@@ -1223,6 +1223,35 @@ extern "rust-intrinsic" {
    /// }
    /// ```
    ///
+    /// However, since pointers (and references) have provenance and integers do not, the following
+    /// program executes UB:


I feel like this needs more comments explaining what happens -- ideally referring to some module-level docs in std::ptr that explain the entire provenance story.

Right now this PR is the first to introduce the word "provenance" into stable docs. I don't think you can just use the term here without any explanation. So either this needs to be reworded to avoid that term, or the libs change here can only land once there's a proper explanation of "provenance" somewhere (e.g., once strict provenance is stable).

Also, burying this in the examples doesn't seem appropriate. There is existing discussion of ptr2int transmutes up above and down below, it's strange that this is entirely disconnected.

Ah, I see. I will not be the one to write the documentation for what provenance is, so we can either land this without added documentation on the basis of what Scott said here: #121282 (comment) or not at all.

I'm going to force-push away the documentation from this PR.

I'll see if I can add some wording to the docs that warn against int2ptr transmutes without talking directly about promotion. That doesn't have to block this PR.

scottmcm · 2024-03-12T00:25:00Z

This is a small change that looks good to me. Miri has confidently called it UB for ages, as mentioned above, and @saethlin has done as much as can be expected from the crater results. Anyone broken by it can change to using a cast instead of the transmute, which is something that doesn't have MSRV issues since casts have existed forever.

So let's do it!
@bors r+ rollup=never

I agree it would be nice to have more documentation about what is and isn't allowed with transmutes and provenance, but that's the existing state of the docs, since misusing this is already UB. The special transmute path is itself relatively new -- I added it in #109843, where arguably I shouldn't have used inttoptr -- and not something that was ever guaranteed to be an inttoptr.

bors · 2024-03-12T00:25:03Z

📌 Commit 2eb9c6d has been approved by scottmcm

It is now in the queue for this repository.

bors · 2024-03-12T04:11:41Z

⌛ Testing commit 2eb9c6d with merge 0fa7fea...

bors · 2024-03-12T06:28:00Z

☀️ Test successful - checks-actions
Approved by: scottmcm
Pushing 0fa7fea to master...

rust-timer · 2024-03-12T07:41:56Z

Finished benchmarking commit (0fa7fea): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.6%	[2.6%, 2.6%]	1
Regressions ❌ (secondary)	2.8%	[2.8%, 2.8%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-7.8%	[-7.8%, -7.8%]	1
All ❌✅ (primary)	2.6%	[2.6%, 2.6%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 672.319s -> 672.448s (0.02%)
Artifact size: 310.04 MiB -> 310.05 MiB (0.00%)

Ignore some more crates Adding all the spurious failures I identified in rust-lang/rust#121282 (comment) I'm also not sure what `blacklist.md` is for, but it hasn't been updated in 6 years so I imagine it's not important.

transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc `@saethlin` `@scottmcm` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc ``@saethlin`` ``@scottmcm`` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc ```@saethlin``` ```@scottmcm``` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc ````@saethlin```` ````@scottmcm```` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc `````@saethlin````` `````@scottmcm````` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

Rollup merge of rust-lang#122379 - RalfJung:int2ptr-transmute, r=m-ou-se transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc ```@saethlin``` ```@scottmcm``` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

transmute: caution against int2ptr transmutation This came up in rust-lang#121282. Cc ```@saethlin``` ```@scottmcm``` Eventually we'll add a proper description of provenance that we can reference, but that's a bunch of work and it's unclear who will have the time to do that when. Meanwhile, let's at least do what we can without mentioning provenance explicitly.

Pkgsrc changes: * Adapt checksums and patches, some have beene intregrated upstream. Upstream chnages: Version 1.78.0 (2024-05-02) =========================== Language -------- - [Stabilize `#[cfg(target_abi = ...)]`] (rust-lang/rust#119590) - [Stabilize the `#[diagnostic]` namespace and `#[diagnostic::on_unimplemented]` attribute] (rust-lang/rust#119888) - [Make async-fn-in-trait implementable with concrete signatures] (rust-lang/rust#120103) - [Make matching on NaN a hard error, and remove the rest of `illegal_floating_point_literal_pattern`] (rust-lang/rust#116284) - [static mut: allow mutable reference to arbitrary types, not just slices and arrays] (rust-lang/rust#117614) - [Extend `invalid_reference_casting` to include references casting to bigger memory layout] (rust-lang/rust#118983) - [Add `non_contiguous_range_endpoints` lint for singleton gaps after exclusive ranges] (rust-lang/rust#118879) - [Add `wasm_c_abi` lint for use of older wasm-bindgen versions] (rust-lang/rust#117918) This lint currently only works when using Cargo. - [Update `indirect_structural_match` and `pointer_structural_match` lints to match RFC] (rust-lang/rust#120423) - [Make non-`PartialEq`-typed consts as patterns a hard error] (rust-lang/rust#120805) - [Split `refining_impl_trait` lint into `_reachable`, `_internal` variants] (rust-lang/rust#121720) - [Remove unnecessary type inference when using associated types inside of higher ranked `where`-bounds] (rust-lang/rust#119849) - [Weaken eager detection of cyclic types during type inference] (rust-lang/rust#119989) - [`trait Trait: Auto {}`: allow upcasting from `dyn Trait` to `dyn Auto`] (rust-lang/rust#119338) Compiler -------- - [Made `INVALID_DOC_ATTRIBUTES` lint deny by default] (rust-lang/rust#111505) - [Increase accuracy of redundant `use` checking] (rust-lang/rust#117772) - [Suggest moving definition if non-found macro_rules! is defined later] (rust-lang/rust#121130) - [Lower transmutes from int to pointer type as gep on null] (rust-lang/rust#121282) Target changes: - [Windows tier 1 targets now require at least Windows 10] (rust-lang/rust#115141) - [Enable CMPXCHG16B, SSE3, SAHF/LAHF and 128-bit Atomics in tier 1 Windows] (rust-lang/rust#120820) - [Add `wasm32-wasip1` tier 2 (without host tools) target] (rust-lang/rust#120468) - [Add `wasm32-wasip2` tier 3 target] (rust-lang/rust#119616) - [Rename `wasm32-wasi-preview1-threads` to `wasm32-wasip1-threads`] (rust-lang/rust#122170) - [Add `arm64ec-pc-windows-msvc` tier 3 target] (rust-lang/rust#119199) - [Add `armv8r-none-eabihf` tier 3 target for the Cortex-R52] (rust-lang/rust#110482) - [Add `loongarch64-unknown-linux-musl` tier 3 target] (rust-lang/rust#121832) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Bump Unicode to version 15.1.0, regenerate tables] (rust-lang/rust#120777) - [Make align_offset, align_to well-behaved in all cases] (rust-lang/rust#121201) - [PartialEq, PartialOrd: document expectations for transitive chains] (rust-lang/rust#115386) - [Optimize away poison guards when std is built with panic=abort] (rust-lang/rust#100603) - [Replace pthread `RwLock` with custom implementation] (rust-lang/rust#110211) - [Implement unwind safety for Condvar on all platforms] (rust-lang/rust#121768) - [Add ASCII fast-path for `char::is_grapheme_extended`] (rust-lang/rust#121138) Stabilized APIs --------------- - [`impl Read for &Stdin`] (https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#impl-Read-for-%26Stdin) - [Accept non `'static` lifetimes for several `std::error::Error` related implementations] (rust-lang/rust#113833) - [Make `impl<Fd: AsFd>` impl take `?Sized`] (rust-lang/rust#114655) - [`impl From<TryReserveError> for io::Error`] (https://doc.rust-lang.org/stable/std/io/struct.Error.html#impl-From%3CTryReserveError%3E-for-Error) These APIs are now stable in const contexts: - [`Barrier::new()`] (https://doc.rust-lang.org/stable/std/sync/struct.Barrier.html#method.new) Cargo ----- - [Stabilize lockfile v4](rust-lang/cargo#12852) - [Respect `rust-version` when generating lockfile] (rust-lang/cargo#12861) - [Control `--charset` via auto-detecting config value] (rust-lang/cargo#13337) - [Support `target.<triple>.rustdocflags` officially] (rust-lang/cargo#13197) - [Stabilize global cache data tracking] (rust-lang/cargo#13492) Misc ---- - [rustdoc: add `--test-builder-wrapper` arg to support wrappers such as RUSTC_WRAPPER when building doctests] (rust-lang/rust#114651) Compatibility Notes ------------------- - [Many unsafe precondition checks now run for user code with debug assertions enabled] (rust-lang/rust#120594) This change helps users catch undefined behavior in their code, though the details of how much is checked are generally not stable. - [riscv only supports split_debuginfo=off for now] (rust-lang/rust#120518) - [Consistently check bounds on hidden types of `impl Trait`] (rust-lang/rust#121679) - [Change equality of higher ranked types to not rely on subtyping] (rust-lang/rust#118247) - [When called, additionally check bounds on normalized function return type] (rust-lang/rust#118882) - [Expand coverage for `arithmetic_overflow` lint] (rust-lang/rust#119432) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Update to LLVM 18](rust-lang/rust#120055) - [Build `rustc` with 1CGU on `x86_64-pc-windows-msvc`] (rust-lang/rust#112267) - [Build `rustc` with 1CGU on `x86_64-apple-darwin`] (rust-lang/rust#112268) - [Introduce `run-make` V2 infrastructure, a `run_make_support` library and port over 2 tests as example] (rust-lang/rust#113026) - [Windows: Implement condvar, mutex and rwlock using futex] (rust-lang/rust#121956)

rustbot assigned fmease Feb 19, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 19, 2024

saethlin force-pushed the gep-null-means-no-provenance branch from 03e5b5d to 903dd09 Compare February 19, 2024 00:50

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Feb 19, 2024

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Feb 19, 2024

craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 19, 2024

saethlin added the T-opsem Relevant to the opsem team label Feb 19, 2024

craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Feb 20, 2024

craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 20, 2024

saethlin mentioned this pull request Feb 21, 2024

Generate getelementptr instead of inttoptr for ptr::invalid #121242

Closed

scottmcm reviewed Feb 26, 2024

View reviewed changes

RalfJung reviewed Mar 11, 2024

View reviewed changes

Lower transmutes from int to pointer type as gep on null

2eb9c6d

saethlin force-pushed the gep-null-means-no-provenance branch from e8e53b8 to 2eb9c6d Compare March 11, 2024 22:19

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 12, 2024

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 12, 2024

bors merged commit 0fa7fea into rust-lang:master Mar 12, 2024
12 checks passed

rustbot added this to the 1.78.0 milestone Mar 12, 2024

RalfJung mentioned this pull request Mar 12, 2024

transmute: caution against int2ptr transmutation #122379

Merged

saethlin deleted the gep-null-means-no-provenance branch March 12, 2024 13:40

RalfJung mentioned this pull request Mar 23, 2024

Rename expose_addr to expose_provenance #122964

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lower transmutes from int to pointer type as gep on null #121282

Lower transmutes from int to pointer type as gep on null #121282

saethlin commented Feb 19, 2024 •

edited

rustbot commented Feb 19, 2024

saethlin commented Feb 19, 2024

This comment has been minimized.

bors commented Feb 19, 2024

bors commented Feb 19, 2024

This comment has been minimized.

rust-timer commented Feb 19, 2024

saethlin commented Feb 19, 2024

craterbot commented Feb 19, 2024

saethlin commented Feb 20, 2024

craterbot commented Feb 20, 2024

saethlin commented Feb 20, 2024

craterbot commented Feb 20, 2024

scottmcm Feb 26, 2024

craterbot commented Feb 28, 2024

RalfJung Mar 11, 2024

RalfJung Mar 11, 2024

RalfJung Mar 11, 2024

RalfJung Mar 11, 2024 •

edited

RalfJung Mar 11, 2024 •

edited

saethlin Mar 11, 2024

RalfJung Mar 12, 2024

scottmcm commented Mar 12, 2024 •

edited

bors commented Mar 12, 2024

bors commented Mar 12, 2024

bors commented Mar 12, 2024

rust-timer commented Mar 12, 2024

	// CHECK: %_0 = getelementptr i8, ptr null, i64 %i
	// CHECK: %_0 = getelementptr i8, ptr null, [[USIZE]] %i

	/// assert_eq!(*ptr, 0);
	/// assert_eq!(*ptr, 0); // undefined behavior! ⚠️

	/// let new_ref = unsafe {
	/// // undefined behavior! ⚠️
	/// let new_ref = unsafe {

Lower transmutes from int to pointer type as gep on null #121282

Lower transmutes from int to pointer type as gep on null #121282

Conversation

saethlin commented Feb 19, 2024 • edited

rustbot commented Feb 19, 2024

saethlin commented Feb 19, 2024

This comment has been minimized.

bors commented Feb 19, 2024

bors commented Feb 19, 2024

This comment has been minimized.

rust-timer commented Feb 19, 2024

Overall result: no relevant changes - no action needed

saethlin commented Feb 19, 2024

craterbot commented Feb 19, 2024

saethlin commented Feb 20, 2024

craterbot commented Feb 20, 2024

saethlin commented Feb 20, 2024

craterbot commented Feb 20, 2024

Choose a reason for hiding this comment

craterbot commented Feb 28, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung Mar 11, 2024 • edited

Choose a reason for hiding this comment

RalfJung Mar 11, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scottmcm commented Mar 12, 2024 • edited

bors commented Mar 12, 2024

bors commented Mar 12, 2024

bors commented Mar 12, 2024

rust-timer commented Mar 12, 2024

Overall result: no relevant changes - no action needed

saethlin commented Feb 19, 2024 •

edited

RalfJung Mar 11, 2024 •

edited

RalfJung Mar 11, 2024 •

edited

scottmcm commented Mar 12, 2024 •

edited