Rebalancing coherence.

[nikomatsakis]

I recently realized that our current trait system contains a forward compatibility hazard concerned with negative reasoning. The TL;DR is that negative reasoning (that is, the compiler saying that "doing X is legal if the trait T is NOT implemented for some type U") can easily make it impossible to add impls of any traits in a backwards compatible way. The most obvious example of negative reasoning are negative trait bounds, which have been proposed in a rather nicely written RFC. However, even without negative bounds, the trait system as currently implemented already has some amount of negative reasoning, in the form of the coherence system.

This RFC is fairly simple proposal that tries to strike a good balance between parent and child crates, in terms of permitting parent crates to expand but also giving child crates lots of freedom to define the impls they need. However, it does involve tightening coherence so it is somewhat more restrictive (the current rules are designed to permit as much as possible in the child crates; but this winds up limiting the parent crates).

Some prior discussion is on this internals thread, although it's mostly me talking to myself.

Rendered view

[edited to link to final rendered version]

[quantheory]

You list three things after this.

[quantheory]

s/vesion/version/

[quantheory]

I don't know that I have thought through the implications enough to be strongly for or against, but I do want to point out that without higher-kinded types, it's sometimes hard to work generically over types of references. This encourages library developers to define interfaces that use mainly &T and &mut T, but if/when HKT lands, using some generic Ref<'a, T> would become more feasible, and possibly more popular for presenting "views" of data. Those objects would presumably have to be fundamental in order to be used similarly to &T.

Question on a different topic: Should Deref be fundamental purely due to the special status it has in the language? I believe that the current rules regarding coercion, autoref, and autoderef for method calls are such that adding a Deref impl is not a breaking change (e.g. we don't bother with an autoderef for a method call if we've already found an applicable impl). But it's not completely obvious to me that this is always the case (or that we want to require this into the indefinite future).

[theemathas]

I like the general idea, but there are some corner cases, especially with #[fundamental]. I will add some line notes for those.

BTW, I believe that the specifics of #[fundamental] should be moved to the Detailed Design section.

[theemathas]

What exactly is a "blanket impl"? Does something like impl<T> SomeTrait for Foo<T, Bar> {...} count?

[theemathas]

Why Sized, but not Deref, DerefMut, Index, or IndexMut?

[theemathas]

What about impl<P1...Pn> Trait<T1...Tn> for T0<U1...Un>?

[theemathas]

Does this allow things like impl<A, B, T: SomeTrait<A, B>> ReverseTrait<B, A> for T?

[nikomatsakis]

@theemathas

Does this allow things like impl<A, B, T: SomeTrait<A, B>> ReverseTrait<B, A> for T?

That is allowed if ReverseTrait is defined in the current crate, but not otherwise. (As today.)

[nikomatsakis]

@theemathas

What about impl<P1...Pn> Trait<T1...Tn> for T0<U1...Un>?

I'm not really sure what you're getting at exactly. In particular, T0 is any type, so it could be things like Box<Foo> and so on as well as simple types like i32.

The main effect of this RFC is to limit the compound types permitted in T0, however. For example, today, something like (Box<LocalType>, i32) would be accepted. Under this RFC, that is not permitted, because tuples are not considered "fundamental". However, Box<LocalType> or &Box<LocalType> or &LocalType is ok, because & and Box are fundamental.

[theemathas]

What about *const LT and *mut LT?

[nikomatsakis]

They are not included in the proposed set of fundamental types.

[nikomatsakis]

Sorry, let me expound. I do not think it is terribly common to need to implement traits for "naked" unsafe pointer types like that. It certainly didn't arise even once in any of the code I looked at. If you wanted to do so, you could make a newtype (which is probably a good idea, because unsafe pointers aren't really a complete abstraction on their own).

[theemathas]

What about type and use items? What about primitive types?

[nikomatsakis]

type and use introduce pure aliases and are irrelevant here. This is focusing on the core type system, in which no aliases exist.

[theemathas]

Does this allow where Tj: SomeTrait<P1>?

[nikomatsakis]

where clauses are not restricted in any way. What is restricted is the use of a where-clauses to permit otherwise overlapping impls. So if you had:

impl<T> Foo for T
    where T: SomeTrait<i32>
{ }

impl Foo for SomeType { }

this is legal only if SomeType: !SomeTrait<i32>. This RFC requires that this negative constraint meet orphan requirements before we consider it satisfied, to take into account the possibility of future impls being added. Basically that means that SomeType or SomeTrait must be local to the current crate (though if SomeTrait is fundamental, the rules are a bit more lenient).

[pnkfelix]

cc me

[nikomatsakis]

Just to answer the question that was asked a few times as to why the specific set of "fundamental" traits was chosen:

1. I'm not sure if Sized is actually needed. I added it early on in the code and I forget why. I'll try taking it out to see what happens.
2. I was trying to resist the temptation to mark a lot of traits as fundamental. I'd prefer to make none. The reason is that every trait we mark as fundamental is an additional constraint that any future generalization has to support. It might be that we find #[fundamental] completely adequate as a scheme for allowing one to balance negative reasoning and possible future expansion, but we may also find some other nifty approach, and I wanted to minimize the set of things that any possible future approach would have to support.

That said, I could certainly see some logic for marking all the "overload" traits (Deref, Fn, Index, Add, etc) as fundamental just because it's a coherent set that should be easy to remember. The fact is that the big step is going from zero fundamental traits to one; going from one to N is somewhat less scary. Still, we can easily expand the set of fundamental traits in a backwards compatible way, so I don't feel a ton of pressure to figure this out this second.

[cristicbz]

I noticed in the alternative sections, you talk about specialization and bring up the problem of associated types (brilliant catch by the way). I'm curious about this paragraph (my emphasis):

For example, I experimented with a "forward proofing" rule which required that, if one impl is more specialized than another according to this definition, then all of their associated type bindings must match. This was an experiment to see whether it was possible to write some kind of "rules" for how associated types should be based on their inputs. Unfortunately, this proved to be untenable. There are numerous trait patterns that we use all the time where this constraint does not hold.

My use case of specialisation, like you mention earlier in that doc, would be to provide optimised implementations, like your zip example. In that case we would certainly want associated types to match (or at least be covariant). Essentially we want it to be transparent that you're using a specialised impl and adding specialisations should not be a breaking change.

I can't think of any of the "numerous trait patterns" that you mention. I'm sure you're actually right, but could you please elaborate for posterity? The specialisation solution would have made me really happy as it seems both less ad-hoc and less restrictive than the solution proposed in the RFC.

[theemathas]

@nikomatsakis The main reason I asked about the overloading traits is that some operators perform autoderef / autoref (to be precise, Index, IndexMut, Fn, FnMut, FnOnce), while Deref and DerefMut control autoderef. I am not sure about the impact of this on method resolution.

PS You still have not answered about "what is a blanket impl".

[aturon]

After much discussion internally, on the internals post, this thread, and the weekly meeting, this RFC has been approved. It is unfortunate to have to move so quickly on a deep change like this, but the existing system of coherence was simply not in a shippable state without it. The design presented here manages to avoid breaking code while not making strong commitments -- the proposed attribute can be left feature-gated, and eventually replaced with an entirely different mechanism or stabilized as-is, depending on our experience.

Tracking issue