Tracking issue for RFC 2043: Add `align_offset` intrinsic (formerly: and `[T]::align_to` function)

[aturon]

This is a tracking issue for the RFC "Add align_offset intrinsic and [T]::align_to function " (rust-lang/rfcs#2043). align_to is stable, so this tracks just align_offset now.

Steps:

• Implement the RFC (align_offset: #43903; align_to: #50319)
• Adjust documentation (see instructions on forge)
• Stabilization PR (see instructions on forge): align_to: #53754

Unresolved questions:

• "produce a lint in case sizeof<T>() % sizeof<U>() != 0 and in case the expansion is not part of a monomorphisation, since in that case align_to is statically known to never be effective

[SimonSapin]

@oli-obk Is it useful to stabilize align_offset before align_to is implemented?

[oli-obk]

Generally yes, because that's the key feature. Align_to is just convenience

[SimonSapin]

Alright, then. Let’s stabilize align_offset and leave this issue open for tracking align_to.

@rfcbot fcp merge

[rfcbot]

Team member @SimonSapin has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Kimundi
• @SimonSapin
• @alexcrichton
• @aturon
• @dtolnay
• @sfackler
• @withoutboats

Concerns:

• motivation-and-ergonomics resolved by #44488 (comment)

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[alexcrichton]

@rfcbot concern motivation-and-ergonomics

The original motivation for these functions (unless it's changed, which it may have!) I believe was for miri and const evaluating various functions. Is that still the main motivation for these functions? If so do we expect that more things to be available on stable once these are stabilized?

I'm a little unclear about how and when such an intrinsic would be used so I'm mostly looking to straighten up my own thinking! In particular the clause "If it is not possible to align the pointer, the implementation returns usize::max_value()." I'm having trouble wrapping my head around in how it'd be expected to get used.

Is the long term plan to make this a const fn but otherwise disallow operations like *mut T as usize in a const fn?

In terms of ergonomics I'd also naively expect a signature like:

pub fn align_to(self, align: usize) -> Option<*mut Self>;

but would that create the same problems it's trying to avoid?

[oli-obk]

Is that still the main motivation for these functions?

yes

If so do we expect that more things to be available on stable once these are stabilized?

This will allow making it const fn, which is a separate topic in the future

I'm having trouble wrapping my head around in how it'd be expected to get used.

No matter which computation you use to figure out the number of bytes that you need to offset your pointer in order to make it aligned, you will need to check whether it still points into your buffer. If you have a [u8; 3] and have a pointer to the first element, how many bytes to do you need to offset in order to get a 8 byte aligned pointer? that might be beyond the range. So you always need to check. Returning max_value means that you can't align it in any buffer.

Is the long term plan to make this a const fn but otherwise disallow operations like *mut T as usize in a const fn?

No, these operations are fine. The issue is that no matter how many bytes you offset a *mut u8 in miri, it will never become aligned to anything with a higher alignment. This function exists to abstract away such manual pointer alignments.

In terms of ergonomics I'd also naively expect a signature like:

You could do that, but then you'd also need an argument for the allocation size. Now that I think about it, that doesn't seem too bad and since you need to check anyway... As long as the intrinsic doesn't have to return an Option (the intrinsic an impl detail anyway).

[alexcrichton]

Ok, thanks for the info! It sounds like this function doesn't necessarily unlock anything in the near term as const-things are still pretty unstable? If that's the case I'm personally tempted to leave these as unstable until we've got the const story more fleshed out to see how they fit into the grand scheme of things

[nagisa]

Oh dear, it has been almost a year since the RFC and it went totally outside my radar, until I really, really needed [T]::align_to for my current use case of… aligning a byte buffer :)

Thinking about possible implementation approaches, I realised that to implement [T]::align_to in some cases it would be necessary to resort to calling the intrinsic multiple times in a manner similar to this:

loop {
    x = intrinsics::align_offset(ptr, align);
    if (x is aligned to object start) { return }
    ptr = x.offset(1);
}

So with @oli-obk we decided that it would probably be most prudent to simply change the intrinsic to do such calculation by itself, changing its signature from the current one to align_offset<T>(*mut T, usize) -> *mut T, that aligns the pointer to the first aligned object, not the first aligned pointer value, like it does currently.

[nagisa]

I’m almost done with the improved implementation of the align_offset intrinsic, and I have noticed, that we currently specify that offset returned by the code is in bytes. This seems suboptimal in multiple ways.

1. Since implementation works strictly in units of "elements", a conversion step is always necessary for when the stride of T does not equal 1;
2. Furthermore, to implement align_to, conversion from bytes back to elements is necessary again;
3. Cannot be used directly with ptr.offset()/ptr.offset_to without converting bytes back into "elements" again;

I propose that we change align_offset to return offset in number of elements.

[oli-obk]

I'm assuming we are talking about elements of the source slice and not the aligned slice. Sgtm

[nagisa]

I guess I should’ve said in “number of Ts” where <*const T>::align_offset(...), but yeah.

[scottmcm]

IIRC this was in bytes in the intrinsic because the pointer to align could have a ZST pointee, and always using bytes sidesteps the questions of what to do there. (Not that I'm sure why you'd want to align a *const [i32; 0] to *const [u64; 0], but you could.) The library could plausibly have a better API, though...

[nagisa]

FYI implementation for align_to and changes to align_offset are at #50319.

[RalfJung]

As another case in point that getting this code right when hand-written is hard: Spot the issue in this code.

The problem is that the inner slice is not aligned properly in all cases when it is empty.

If we had a stable align_to, they could just use that. :)

[oli-obk]

ping @llogiq

[llogiq]

This shouldn't happen because we only call the function for larger slices. But I've been wrong in the past, so I wouldn't bet on it.

[nagisa]

I’d like to suggest to focus on stabilising [T]::align_to if we are not sure whether align_offset is a right API for us. [T]::align_to seems to me like something that has provably the ideal API for what it does and covers majority of the use cases of the bare-bones align_offset.

The fact that it took me (with a help from another person) months to implement align_to has something to say about non-triviality of such function.

[RalfJung]

Agreed, the sooner we can get people to use this instead of hand-rolling their own, the better. :)

Just one comment on a remark in the implementation PR:

Furthermore, a promise is made that the slice containing Us will be as large as possible (contrary to the RFC) – otherwise the function is quite useless.

miri cannot guarantee this. So while the implementation should do that, users of this code should NOT rely on the U part being maximal.

I cannot think of a case where this is require for correctness anyway, is there one?

[nagisa]

Oh yes, that is true. I’m fine with removing that guarantee from the documentation.

[RalfJung]

So, what would this take? Just open an RFC that stabilizes, and ask for FCP?

[scottmcm]

@RalfJung Yeah, I think a PR to stabilize (with a comment about why) is sufficient.

[RalfJung]

PR submitted: #53754

[SimonSapin]

<*mut u8>::align_offset can be useful when implementing a memory allocator, to satisfy the requested layout.align(). Note however that much of the complexity of the implementation of <*mut T>::align_offset does not apply when size_of::<T>() == 1. The function is also infallible in that case.

@alexcrichton Would a separate function or method for this special case (or maybe even for usize instead of pointers) address your concern?

[alexcrichton]

@SimonSapin yeah the API I mentioned above I think would be fine to add and consider stabilizing

[TimDiekmann]

Is it has the disposition-merge tag since 9 months, which issues needs to be resolved before solving this?

<*mut u8>::align_offset can be useful when implementing a memory allocator, to satisfy the requested layout.align().

This is exactly my use case.

[nagisa]

There is still an outstanding concern from @alexcrichton about the ergonomics of the API.

[RalfJung]

Uh, didn't it get stabilized with #53754 ?

EDIT: Oh, I guess this tracking issue is about align_offset only now?

[nagisa]

Oh, I guess this tracking issue is about align_offset only now?

Yes.

[alexcrichton]

@rfcbot resolve motivation-and-ergonomics

I don't want to personally be on the hook for blocking this any more

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.