Tracking issue for DST coercions (coerce_unsized, unsize) stabilization

[aturon]

DST coercions were the last part of DST to land, and are still undergoing some changes.

Before stabilization, at the very least the API surface here needs some scrutiny from the libs team for naming and other conventions issues.

cc @nrc @gankro

Things to consider prior to stabilization

• Interaction with pin, see this internals thread and Pin is unsound due to transitive effects of CoerceUnsized #68015 for details.

[aturon]

@nrc, what's your sense of the status for these features? Do you feel ready to stabilize after the libs team takes a look?

[nrc]

My feeling (and it is really just a feeling, I have no data to back this up) is that DST coercions haven't yet got the use they need to prove themselves for stabilisation. I'd love to be wrong about this - if someone could show me a bunch of code doing stuff with DST coercions I'd be very happy. OTOH, I don't think there is anything fundamentally wrong with them, so it would not be a waste of time for the libs team to take a look.

[photino]

Why is the marker named Unsize? Unsized sounds better, since we already have Sized.

[arielb1]

@photino

It goes the wrong way around - for example, we have [i8; 2]: Unsize<[u8]> (and also [i8; 2]: Unsize<fmt::Debug>). We can swap the order of parameters - U: Unsized<T> if U is thean unsized version of T.

[rphmeier]

Can we move to stabilize this? It's been unchanged for 5 months and I've heard no complaints about it.

[Gankra]

I don't recall having any issues with this in libstd. The correct way to use this is basically a witchcraft incantation (mostly because these are under-the-covers things one never interacts with in real code), but as far as I know, there's know way to use it dangerously wrong? The whole thing Just Works.

I guess one thing I've never really thought about is why one needs to talk about whether the pointed-to value is unsizeable or whatever. Why can't one simply declare that pointer X is an unsizable one, and have all that junk automatic? Is this desirable to have for generic consumers...?

[jnicholls]

It's desirable for generic consumers of e.g. Arena allocators that allocate a Sized type that implements a trait unknown to the allocator, and the consumer wants to coerce the allocated result to an allocated trait object.

See https://gitlab.com/Bastacyclop/rust_arena/blob/master/src/lib.rs in the tests. It's a decent use case. How would you recommend doing this otherwise?

[stevenblenkinsop]

Is the lack of support for enums deliberate, temporary, or merely overlooked?

[aturon]

I'm nominating this for discussion at the next lang team meeting.

[Gankra]

@jnicholls maybe I'm missing something, but just mark the one pointer field that will be Unsized as coercable: https://gitlab.com/Bastacyclop/rust_arena/blob/master/src/lib.rs#L41

It seems to me that all the impl<...> CoerceUnsized for ... is trivially derivable from the claim that that field should be coercable.

[jnicholls]

@gankro Sorry I don't follow what you mean. Can you provide an example?

[Gankra]

Oops, something clobbered my selection, I meant to link to: https://gitlab.com/Bastacyclop/rust_arena/blob/master/src/lib.rs#L59

Basically instead of

pub struct Allocated<'a, 'b, T: 'b + ?Sized> {
    reference: &'b mut T,
    phantom: PhantomData<&'a ()>
}

impl<'a, 'b, T: ?Sized, U: ?Sized> CoerceUnsized<Allocated<'a, 'b, U>> for Allocated<'a, 'b, T>
    where U: 'b, T: 'b + Unsize<U> {}

You would just do something like:

#[coerce_unsized(T)]
pub struct Allocated<'a, 'b, T: 'b + ?Sized> {
    reference: &'b mut T,
    phantom: PhantomData<&'a ()>
}

[aturon]

cc @eddyb

[aturon]

Removed T-libs tag, but note that we'd like to review the naming conventions here before stabilization.

[eddyb]

There's room for generalization and a good chance we might want Source: Coerce<Target>.
See this comment in response to wanting &mut T reborrows working through structs.

[apasel422]

Right now, because T does not implement Unsize<T>, one cannot use CoerceUnsized to support optional coercions in a generic context. (My use case is supporting DSTs with the placement API). Since &T can trivially be coerced to &T, would it make sense to add the identity impl?

[tomaka]

I'm not very familiar with how this works, but shouldn't the T of CoerceUnsized be T: ?Sized?
Right now T is required to be Sized, which doesn't make sense for me. Example.

[arielb1]

@tomaka

You implement CoerceUnsized on pointer types, not on DST types.

e.g.

impl<'a, T: ?Sized+Unsize<U>, U: ?Sized> CoerceUnsized<&'a mut U> for &'a mut T {}

[alexcrichton]

(adding T-libs as this has a library API associated with it which needs stabilization)

[nikomatsakis]

So at a recent @rust-lang/lang meeting we discussed this. I think everyone felt...somewhat comfortable with the idea of going forward, but we also wanted to do a thorough review of the current status of the code, along with a comparison against the existing specifications, to make sure everything is well aligned.

[eddyb]

@nikomatsakis The problem, though, is that the RFC is intentionally provisional, mostly to get Rc and Arc working with DSTs. I would like to see an attempt at generalization before stabilizing it (I believe the compiler code would "just work" with more than one field being coerced, and various coercion modes, not just Unsize, with minimal changes).

Maybe with specialization we can even encode Deref coercions in a general Coerce trait.

[nikomatsakis]

@eddyb we discussed in @rust-lang/lang meeting today -- splitting off into two feature gates makes sense, provided we have a good description of what the two gates cover and the interaction between them. (e.g., as the issue comments)

[matthieu-m]

Has any consideration been given to having T: Unsize<T> for Sized types?

When attempting to use Unsize in generic contexts, such as in this example, the inability to abstract over sized and unsized types requires splitting APIs in 2, with the same code in each, just because T: !Unsize<T>.

[Chronos-Sk]

@matthieu-m That playground example is currently throwing an ICE, in case that's new. Error gist.

[matthieu-m]

@matthieu-m That playground example is currently throwing an ICE, in case that's new. Error gist.

I think this is new, but as I mentioned because T: !Unsize<T> it didn't compile in the first place.

[eggyal]

Currently, there's no std/core type for a "sized str" (by which I mean an array of UTF-8 bytes). Of course, one can create a custom version:

use core::str;

#[repr(transparent)]
pub struct Str<const N: usize>([u8; N]);

impl<const N: usize> Str<N> {
    pub fn new(s: [u8; N]) -> Result<Self, str::Utf8Error> {
        str::from_utf8(&s)?;
        Ok(Self(s))
    }

    pub unsafe fn new_unchecked(s: [u8; N]) -> Self {
        Self(s)
    }
}

But such a custom type cannot be "unsized" to str. Shouldn't there be a library type like this, but that implements Unsize<str>?

[bjorn3]

impl<const N: usize> Deref for Str<N> {
    type Target = str;
    
    fn deref(&self) -> &str {
        unsafe {
            str::from_utf8_unchecked(&self.0)
        }
    }
}

allows you to "unsize" &Str<N> to &str using &*s.

[eggyal]

@bjorn3 Of course, but that's not a coercion (the point of this feature, no?). In particular, given s: Str<N>, one cannot write &s as &str—and hence when coercing generic T: Unsize<U> with &t as &U we cannot provide a Str<N> for t.

[JanBeh]

I have a problem where I want to be generic over reference-like types, and I feel like the current mechanism doesn't allow me something what I would like to do.

[RReverser]

one cannot write &s as &str

Yes, they can, that's what Deref provides. Your example of &s as &str compiles just fine: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=bd56db69728b59c755a066c9e42ef37c

[eggyal]

Yes, they can, that's what Deref provides.

Apologies, perhaps my previous comment could have been clearer: a deref coercion is not an unsize coercion. What one cannot do (without either some relaxation of coherence rules or else Str being in the standard library together with an implementation of Unsize<str>) is use Str<N> in a generic context that will perform an unsize coercion (ie where an implementation of Unsize<str> is required), for example:

fn generic<U: ?Sized>(t: impl Unsize<U>) {
    todo!() // something involving &t as &U
}

fn foo<const N: usize>(s: Str<N>) {
    generic::<str>(s); // Str<N> does not implement Unsize<str>
}

For context, this is what prevents strings being "thinnable" in my (experimental) thinnable crate.

[dest1n1s]

Is there something wrong with the rule

Types implementing a trait Trait also implement Unsize<dyn Trait>.

when it works with a trait object with associate type, such as dyn Trait<Assoc=Type>?

For example:

trait Foo {
    fn foo(&self);
}

trait Bar: Foo {
    type Assoc;

    fn bar(&self);
}

fn upcast<Dyn>(bar: &Dyn) -> &dyn Foo
where
    Dyn: ?Sized + Unsize<dyn Foo>,
{
    bar
}

impl Foo for i32 {
    fn foo(&self) {
        println!("foo");
    }
}

impl Bar for i32 {
    type Assoc = i32;

    fn bar(&self) {
        println!("bar");
    }
}

fn main() {
    let bar: &dyn Bar<Assoc = i32> = &42;
    let foo = upcast(bar);
    foo.foo();
}

in which the compiler says

the trait Unsize<(dyn Foo + 'static)> is not implemented for dyn Bar<Assoc = i32>

But if I try to manually impl the trait Foo for dyn Bar<Assoc=i32>, it says

(dyn Bar<Assoc = i32> + 'static) automatically implements trait Foo

Here's the minimal example: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=85925cd4977fc95beb52ce9a03653755 ,

[Chronos-Sk]

FWIW, that's effectively relying on the incomplete feature trait_upcasting (which seems to have the same difficulty: playground).

[nikomatsakis]

Closing in favor of #18598 -- they seem to cover the same ground.