Allow overlapping implementations for marker traits

[sgrif]

rendered

[arielb1]

The currently presented version of specialization won't allow this. Your proposal is basically lattice impls for marker traits.

[aturon]

@sgrif First, sorry for the very overdue response; I'm behind on my RFC shepherding.

Second: some improvements in this area would be most welcome. For example, the design I recently proposed for a trait here won't work without some adjustment to coherence for marker traits.

Simply allowing overlap for marker traits (as proposed here) seems like a pure win. But I wonder if you've thought much about negative impls and potential overlap there? For example, in the comment I linked above, I propose something like:

use std::cell::UnsafeCell;

trait ExnSafe {}
impl ExnSafe for .. {}

trait RefExnSafe {}
impl RefExnSafe for .. {}

impl<'a, T: RefExnSafe> ExnSafe for &'a T {}

impl<'a, T> !ExnSafe for &'a mut T {}
impl<T> !ExnSafe for UnsafeCell<T> {}

impl<T: 'static + Send> ExnSafe for T {}

// Imagine a variant of RefCell that did poisoning:
impl<T> ExnSafe for PoisoningRefCell<T> {}

struct AssertExnSafe<T>(T);
impl<T> ExnSafe for AssertExnSafe<T> {}
impl<T> Deref for AssertExnSafe<T> { ... }
impl<T> DerefMut for AssertExnSafe<T> { ... }

fn recover<F: FnOnce() -> R + ExnSafe, R>(f: F) -> thread::Result<R>

which has potential for overlap between positive and negative cases. How should that work out?

I've been thinking recently about a more flexible form of specialization, which @arielb1 calls "lattice impls". Whether we go that way or not, I think that we want to provide some way of resolving such ambiguities with specialization, and maybe that's the answer for these conflicts.

Regardless, I don't see a lot of downside to moving forward with the simple case of "positive impl" overlap proposed here in the meantime!

[gereeter]

I'm not certain of what exact restrictions should be in place, but I'm strongly in favor of something like this as it would allow #91 to be implemented as a library.

[sgrif]

I hadn't thought too closely about negative impl overlap, but I started working on actually implementing this, and it turns out we do have an explicit test case which got me thinking about it, and illustrates it pretty well.

I think the right answer here is to disallow any overlap between a negative impl and a positive impl (excluding impl for ..). Once specialization lands, those rules can apply in a straightforward and consistent way, where overlap is permitted as long as one impl is strictly more specific than the other. I'll amend the RFC to be clear on this

[aturon]

We talked about this a bit at last week's lang team meeting, and I think everyone is on board, except for the definition of "marker trait".

The consensus was that, for the purposes of this RFC, coherence rules should be relaxed for any trait that does not contain any items, regardless of what it inherits from.

In particular, inheriting from a trait that happens to include items doesn't actually introduce coherence problems around the subtrait implementation.

Can you update the RFC accordingly? Once you've done that, we can move to final comment period and try to land this.

[sgrif]

I've updated the RFC to use that definition of marker trait

[aturon]

This RFC is entering its Final Comment Period.

[glaebhoerl]

In particular, inheriting from a trait that happens to include items doesn't actually introduce coherence problems around the subtrait implementation.

I'm not sure of the right way to interpret this. As far as I can tell, empty traits with non-empty supertraits couldn't possibly have incoherent impls, because to implement a trait you need to implement its supertraits, and you couldn't impl the non-empty supertrait incoherently. Is this the same thing as what you meant to imply?

[aturon]

@glaebhoerl Yes; thanks for putting it much more clearly :)

[glaebhoerl]

That said, I think you could actually implement a subtrait incoherently even without doing so for the supertrait (which may also be part of what you meant). Contrived example:

trait A { fn f() -> bool; }

impl<T> A for Option<T> { ... }

trait B: A { }

impl<T: Debug> B for Option<T> { }

impl<T: Display> B for Option<T> { }

But this also seems fine.

[arielb1]

Was there an analysis of the interaction with the other parts of the type system, and with the interaction with the current implementation strategy (I think this would basically require the new trait system).

[aturon]

@glaebhoerl Yes, that's what I'm trying to get at: supertraits are simply irrelevant when it comes to the coherence rules of a subtrait, because the impl blocks are separate; we can govern them by separate coherence rules. In particular, if a subtrait doesn't have any items, than there's nothing to be coherent about, which is of course what this RFC is getting at. Hence it makes sense for relaxed coherence rules to apply.

@arielb1 Could you be slightly more concrete/specific? In terms of semantic interactions, this proposal doesn't seem problematic, given that the coherence property is basically vacuous in these cases.

[arielb1]

@aturon

I want to find any surprises before, not after, this is stabilized, especially as I am not sure what is the best way to stabilize this.

[aturon]

@arielb1 I'm definitely on board with that! But what I mean is, I'm not sure if you have any concrete worries in mind? Certainly both @nikomatsakis and I have given this RFC some thought in terms of semantic interactions and haven't turned up anything problematic, but you're often a little more devious, so maybe you see something fishy?

[arielb1]

@aturon

Nothing particularly, except that I don't feel like we understand the current system well enough. On the other hand, this is both not stronger than lattice impls (which we may want), and feels to be semantically equivalent to fixing rust-lang/rust#27544, which we (probably) want to.

[arielb1]

For example, there was the OIBIT = negative impls interaction.

[arielb1]

One thing that worries me is that we could have non-deterministic effects about which impl gets used when inference, particularly region inference, is involved.

impl<'a, 'b> Marker<'a> for &'b T {}
impl<'a> Marker<'a> for T where T: 'a {}

[nikomatsakis]

In discussion in the lang-team meeting today yesterday we decided not to decide anything just yet, since some of us (cough me cough) haven't really taken the time they want to think this through one last time (in particular I wanted to think over what @arielb1 wrote).

[nikomatsakis]

So I've been thinking this over. Certainly the implementation strategy is a bit unclear (as @arielb1 hinted at earlier). Simply lifting the coherence restrictions is easy enough, but we will encounter some challenges when we come to test whether a given trait impl holds. For example, if we have something like:

impl<T:Send> MarkerTrait for T { }
impl<T:Sync> MarkerTrait for T { }

means that a type Foo: MarkerTrait can hold either by Foo: Send or by Foo: Sync. Today, we prefer to break down an obligation like Foo: MarkerTrait into component obligations (e.g., Foo: Send). Due to coherence, there is always one best way to do this (sort of --- where clauses complicate matters). That is, except for complications due to type inference, there is a best impl to choose. But under this proposal, there would not be.

This is not necessarily a reason not to accept this RFC, but it is a reason to tread cautiously. I've also encountered similar situations when exploring possible extensions to the specialization RFC. Similar problems arise due to where-clauses/projections (e.g. rust-lang/rust#20297) and also regions (e.g., rust-lang/rust#21974). So I definitely would like to make progress here on improving our impl and search strategy in general.

[nikomatsakis]

Another point which arose is that the actual motivation for this RFC is somewhat thin. I feel like I can recall various examples where this has arisen in the past, but it'd be nice to document some of them better. Certainly Gabhor's transmute proposal #91 (as @gereeter pointed out) is an example. I suppose ExnSafe is another example, though perhaps a bit more elaboration is needed there (particularly around negative impls, which is somewhat of an orthogonal situation).

[nikomatsakis]

Huzzah! The language design subteam has decided to accept this RFC. I plan to add a small note to the motivation concern the ExnSafe sort of use-case. Also, the implementation concerns are still very real -- it seems however that we expect to address many of the same questions as part of the specialization work, so hopefully we will find a nice solution. (If not, we'll rollback the RFC.)

[nikomatsakis]

Tracking issue is rust-lang/rust#29864

[sgrif]

Thanks! Sorry for the delay on following up

[nikomatsakis]

I accidentally rebased when I landed this, so I am going to close the PR manually.

[derekdreery]

Could you update the link at the top now it's been merged 😁

[Centril]

Done.

[derekdreery]

Thanks very much :D