Is a read() of the last byte in the address space allowed?

[RalfJung]

Can it ever be fine to do a read() that covers the last byte of the address space? On their own there is nothing wrong with such reads, but allowing them means the following property is violated: if p.read() is fine, then p.add(1) is also well-defined.

@chorman0773 mentioned this property as desirable. I agree it is nice, though I don't see any concrete benefits. Is it desirable enough to introduce extra UB?

Note: it is already clear that having a &i32 cover the last byte of the address space violates the safety invariant. You can turn this into a 1-element array in-place and then iterate over it, which will do the equivalent of p.add(1). Therefore, an allocator that returns !0 for a 1-byte allocation request must already be considered invalid (or else every single user would have to check this before giving that raw pointer to some safe code as a reference).

[chorman0773]

IMO, &T over the last byte of the address space violates the validity invariant, because you can cause UB using only safe language operations - specifically the #[repr(C)] struct Foo{x: i32, y: ()} example gives you a null reference (or wraps the address space) for &foo.y. I do not believe we have any other examples of safe language operations causing UB from violating a safety invariant (str match does not cause UB to my knowledge since you cannot exhaustively match a str).

From the above, we also lose the property that producing a &T from a *const T, is not the same as reading it. AFAIK, retags enforce all reference invariants by virtue of existing, so this wouldn't be satisfied for the above validity invariant.

[RalfJung]

IMO, &T over the last byte of the address space violates the validity invariant, because you can cause UB using only safe language operations

That's exactly what the safety invariant is about: ensure well-definedness of all safe operations. Hence the name. Having a &&&bool that points to invalid data is also "just" a safety invariant violation, and yet can cause UB with only safe language operations.

So this is not an argument to have a validity invariant here.

[chorman0773]

That's exactly what the safety invariant is about. Having a &&&bool that points to invalid data is also "just" a safety invariant violation, and yet can cause UB with only safe language operations.

Yeah, i thought about recursive (in)validity. However, I'd argue there's a validity invariant violation at some level, since every way that causes UB with safe language op involves doing a typed copy as type bool AFAIK, so that would still favor this being a validity invariant.

[RalfJung]

FWIW, the Miri logic for ensuring that reads are in-bounds already says that the end ptr must not overflow.

In MiniRust, the allocator will not create allocations where base+len overflows, so this is implicitly an invariant on all allocations that exist.

---

Yeah, i thought about recursive (in)validity. However, I'd argue there's a validity invariant violation at some level, since every way that causes UB with safe language op involves doing a typed copy as type bool AFAIK, so that would still favor this being a validity invariant.

I can't tell if you are trying to redefine what "validity invariant" means or if you are suggesting that creating an &&&bool where the invalid data is behind references (but the immediate data is fine) should be insta-UB, but I object to both of these things.

[chorman0773]

I can't tell if you are trying to redefine what "validity invariant" means or if you are suggesting that creating an &&&bool where the invalid data is behind references (but the immediate data is fine) should be insta-UB, but I object to both of these things.

No, I'm simply observing that as far as I'm aware, to cause UB using &&&bool in safe language operations, the operation invovles doing a typed copy as bool, which asserts the validity invariant of bool. ***r is an explicit typed copy, and match r{ true => {}, false => {} } I believe does a typed copy to evaluate the patterns. I am unaware of any other way to cause UB with a &&&bool without recursive reference UB.

[RalfJung]

Sure, but it remains a fact that &&&bool triple-pointing to valid data is a safety invariant. If it was a validity invariant, this would be UB:

let x: &&&bool = transmute(&&&2u8);

Also what does this have to do with this issue?

[the8472]

So this is not an argument to have a validity invariant here.

Though we would make it one if we exploited the "references to the top byte are illegal" property to add niches to references since the only things that can actually point there are raw pointers.

[RalfJung]

Even if references had such a validity invariant, that would not necessarily imply anything about rawptr.read().

[digama0]

I think we could make read() (mostly) invalid indirectly, by saying that allocators can't allocate the last byte and read() can only read allocated bytes. I am not entirely sure what the rules here are around external allocations: they are usually accessed using exposed pointers with explicit numeral addresses (i.e. invalid()) , and I don't think there is any other way to get a pointer which does not pass through a reference at some point.

So this reduces to only one case: is std::ptr::read(!0usize as *const u8) valid, assuming there is an external allocation at that address? And I would be okay with the answer being "yes" to this, unless there are specific LLVM reasons why we can't. (The answer "no" on the basis of some extra theorems also seems fine, so maybe this isn't helpful since it seems to be the main question.)

[saethlin]

with explicit numeral addresses (i.e. invalid())

You mean from_exposed right?

[digama0]

either one would work AFAIK, for an exposed allocation

EDIT: Actually now I'm not sure, maybe you are right and using invalid would be UB even on an external allocation.

[RalfJung]

Yes, invalid() is definitely UB to load from (if the load has non-zero size).

[RalfJung]

I think we have a fairly clear answer to this now: the last byte of the address space can never be inbounds of an allocation, so read on that address is always UB.

However, volatile_read can be used to access memory that exists outside the Rust Abstract Machine, including the addresses 0 and usize::MAX.