Skip to content

Commit

Permalink
Rollup merge of #82751 - RalfJung:offset_from, r=dtolnay
Browse files Browse the repository at this point in the history
improve offset_from docs

`@thomcc` pointed out that the current docs leave it kind of unclear how one can satisfy the "no wrapping around `isize` or the address space" requirement of `offset_from`, so make the docs clearer about that.

FWIW, I don't think I entirely agree with that second paragraph about large objects (that I left mostly unchanged here). LLVM, to my knowledge, fundamentally assumes that all allocations fit into an `isize::MAX`. So in that sense creating a larger allocation is simply UB. I would expect a guarantee that Rust heap allocation methods will never return allocations larger than `isize::MAX` (or rather, Rust heap allocation methods should require that the `Layout` is no larger than `isize::MAX`). However, I cannot find any such requirement documented currently. Large allocations are not mentioned at all in the allocator docs, which is quite surprising -- even if we say that such allocations are not insta-UB (which I think is incompatible with LLVM), they are still extremely footgunny since `ptr::offset`/`ptr::add` do not support offsetting by more than `isize::MAX` bytes.

Furthermore, the allocator docs don't even say anything about allocations wrapping around the address space. But that is certainly something allocators must ensure never happens; we cannot expect clients to defend against this.

Cc `@rust-lang/wg-allocators`
  • Loading branch information
JohnTitor committed Mar 7, 2021
2 parents a5a825e + ebe0407 commit 05a2366
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 14 deletions.
20 changes: 13 additions & 7 deletions library/core/src/ptr/const_ptr.rs
Original file line number Diff line number Diff line change
Expand Up @@ -320,25 +320,31 @@ impl<T: ?Sized> *const T {
/// * Both pointers must be *derived from* a pointer to the same object.
/// (See below for an example.)
///
/// * The distance between the pointers, **in bytes**, cannot overflow an `isize`.
///
/// * The distance between the pointers, in bytes, must be an exact multiple
/// of the size of `T`.
///
/// * The distance between the pointers, **in bytes**, cannot overflow an `isize`.
///
/// * The distance being in bounds cannot rely on "wrapping around" the address space.
///
/// The compiler and standard library generally try to ensure allocations
/// never reach a size where an offset is a concern. For instance, `Vec`
/// and `Box` ensure they never allocate more than `isize::MAX` bytes, so
/// `ptr_into_vec.offset_from(vec.as_ptr())` is always safe.
/// Rust types are never larger than `isize::MAX` and Rust allocations never wrap around the
/// address space, so two pointers within some value of any Rust type `T` will always satisfy
/// the last two conditions. The standard library also generally ensures that allocations
/// never reach a size where an offset is a concern. For instance, `Vec` and `Box` ensure they
/// never allocate more than `isize::MAX` bytes, so `ptr_into_vec.offset_from(vec.as_ptr())`
/// always satisfies the last two conditions.
///
/// Most platforms fundamentally can't even construct such an allocation.
/// Most platforms fundamentally can't even construct such a large allocation.
/// For instance, no known 64-bit platform can ever serve a request
/// for 2<sup>63</sup> bytes due to page-table limitations or splitting the address space.
/// However, some 32-bit and 16-bit platforms may successfully serve a request for
/// more than `isize::MAX` bytes with things like Physical Address
/// Extension. As such, memory acquired directly from allocators or memory
/// mapped files *may* be too large to handle with this function.
/// (Note that [`offset`] and [`add`] also have a similar limitation and hence cannot be used on
/// such large allocations either.)
///
/// [`add`]: #method.add
///
/// # Panics
///
Expand Down
20 changes: 13 additions & 7 deletions library/core/src/ptr/mut_ptr.rs
Original file line number Diff line number Diff line change
Expand Up @@ -491,25 +491,31 @@ impl<T: ?Sized> *mut T {
/// * Both pointers must be *derived from* a pointer to the same object.
/// (See below for an example.)
///
/// * The distance between the pointers, **in bytes**, cannot overflow an `isize`.
///
/// * The distance between the pointers, in bytes, must be an exact multiple
/// of the size of `T`.
///
/// * The distance between the pointers, **in bytes**, cannot overflow an `isize`.
///
/// * The distance being in bounds cannot rely on "wrapping around" the address space.
///
/// The compiler and standard library generally try to ensure allocations
/// never reach a size where an offset is a concern. For instance, `Vec`
/// and `Box` ensure they never allocate more than `isize::MAX` bytes, so
/// `ptr_into_vec.offset_from(vec.as_ptr())` is always safe.
/// Rust types are never larger than `isize::MAX` and Rust allocations never wrap around the
/// address space, so two pointers within some value of any Rust type `T` will always satisfy
/// the last two conditions. The standard library also generally ensures that allocations
/// never reach a size where an offset is a concern. For instance, `Vec` and `Box` ensure they
/// never allocate more than `isize::MAX` bytes, so `ptr_into_vec.offset_from(vec.as_ptr())`
/// always satisfies the last two conditions.
///
/// Most platforms fundamentally can't even construct such an allocation.
/// Most platforms fundamentally can't even construct such a large allocation.
/// For instance, no known 64-bit platform can ever serve a request
/// for 2<sup>63</sup> bytes due to page-table limitations or splitting the address space.
/// However, some 32-bit and 16-bit platforms may successfully serve a request for
/// more than `isize::MAX` bytes with things like Physical Address
/// Extension. As such, memory acquired directly from allocators or memory
/// mapped files *may* be too large to handle with this function.
/// (Note that [`offset`] and [`add`] also have a similar limitation and hence cannot be used on
/// such large allocations either.)
///
/// [`add`]: #method.add
///
/// # Panics
///
Expand Down

0 comments on commit 05a2366

Please sign in to comment.