Meeting proposal: inbounds requirements for offset (and potentially place projections)

[RalfJung]

Summary

Currently, ptr.offset requires the input and output pointer to be in-bounds of the same live allocation (as in, stack variable / heap allocation / static). This creates subtle open questions around offset(0) and introduces a notion of "bounds" for a pointer that is separate from provenance (provenance might restrict a pointer to only be useful for a subrange of an allocation, but the actual allocation bounds are still relevant for offset).

If we weaken the semantics of place expressions to not have a "validity invariant" (see rust-lang/unsafe-code-guidelines#418), the semantics of offset could also become relevant for place projections, so the inbounds requirements would then be the reason why addr_of!(ptr::null::<Type>().field) is UB.

There is an obvious alternative here: make offset (and place projections, if they follow offset semantics) just require "no overflowing the edges of the address space". I'd like to gauge the team's opinion on this question. In particular, one problem here is that LLVM currently has no good way to express "offset that does not wrap": the only option is a plain getelementptr, which loses the "nowrap" information. If we have team consensus that "nowrap" is the semantics we want, maybe that would be a good starting point for organizing some work on the LLVM side to add support for getelementptr nowrap. (We might hence want to invite some LLVM people to this meeting.)

(There are other alternatives, e.g. getelementptr inbounds in LLVM actually allows arithmetic that is inbounds of a freed allocation; we could expose that in Rust.)

This does have some overlap with #9, but we can separate the discussion by just talking about offset/wrapping_offset semantics in #9 when defining place expression evaluation; we don't have to know whether the offset requirements will remain as they are or should be weakened.

Reading

• Should raw pointer deref/projections have to be in-bounds? unsafe-code-guidelines#319
• Can we weaken the requirements for offset? (Was: Should we / can we make all "getelementptr inbounds" into "getelementptr nowrap"?) unsafe-code-guidelines#350
• Validity invariant of places unsafe-code-guidelines#418

Comment policy

These issues are meant to be used as an "announcements channel" regarding the proposal, and not as a
place to discuss the technical details. Feel free to subscribe to updates. We'll post comments when
reviewing the proposal in meetings or making a scheduling decision. In the meantime, if you have
questions or ideas, ping the proposers on Zulip (or elsewhere).

[RalfJung]

Summary of the discussion:

• Connor brings up a potential optimization that Rust's offset (but not LLVM's getelementptr inbounds) permits: deducing that the range between old and new pointer is dereferenceable, and using that to prefetch memory.
• The counterpoint is that in particular for place projections, people often don't expect the inbounds requirement, and accidentally cause UB. For offset this is less common due to the detailed docs, but we'd rather not have place projection be a third offset operation distinct from wrapping_offset and offset.
• The counter-counterpoint is that this mostly came up when people tried to hand-write offset_of, and hopefully that won't happen any more when offset_of is natively supported.
• Overall we have fairly concrete evidence of people not expecting the UB, but only in code that hopefully won't be written any more, vs a very tentative idea that this could be useful for optimizations.
• The general conclusion was that for now it doesn't seem clear that dropping the inbounds is the best call. There was consensus that offset(0) should be allowed always, so we will try to see if LLVM can commit to supporting this. Also we should keep our eyes open for real-world UB caused by the inbounds requirement that does not fall in the category of "hand-rolled offset_of implementation".

[RalfJung]

The general conclusion was that for now it doesn't seem clear that dropping the inbounds is the best call. There was consensus that offset(0) should be allowed always, so we will try to see if LLVM can commit to supporting this.

That turns out to be more tricky than anticipated.

[RalfJung]

Anyway, the meeting has been had, so we can close this.