Tracking issue for specialization (RFC 1210)

[nikomatsakis]

This is a tracking issue for specialization (rust-lang/rfcs#1210).

Major implementation steps:

• Land #30652 =)
• Restrictions around lifetime dispatch (currently a soundness hole)
• default impl (#37653)
• Integration with associated consts
• Bounds not always properly enforced (#33017)
• Should we permit empty impls if parent has no default members? #48444
• implement "always applicable" impls #48538

Unresolved questions from the RFC:

• Should associated type be specializable at all?
• When should projection reveal a default type? Never during typeck? Or when monomorphic?
• Should default trait items be considered default (i.e. specializable)?
• Should we have default impl (where all items are default) or partial impl (where default is opt-in)
• How should we deal with lifetime dispatchability?

[aturon]

Some additional open questions:

• Should we revisit the orphan rules in the light of specialization? Are there ways to make things more flexible now?
• Should we extend the "chain rule" in the RFC to something more expressive, like the so-called "lattice rule"?
• Related to both of the above, how does negative reasoning fit into the story? Can we recover the negative reasoning we need by a clever enough use of specialization/orphan rules, or should we make it more first-class?

[arielb1]

I am not sure that specialization changes the orphan rules:

• The "linking" orphan rules must stay the same, because otherwise you would not have safe linking.
• I don't think the "future compatibility" orphan rules should change. Adding a non-specializable impl under you would still be a breaking change.

Worse than that, the "future compatibility" orphan rules keep cross-crate specialization under pretty heavy control. Without them, default-impls leaving their methods open becomes much worse.

I never liked explicit negative reasoning. I think the total negative reasoning specialization provides is a nice compromise.

[sgrif]

Should this impl be allowed with specialization as implemented? Or am I missing something?
http://is.gd/3Ul0pe

[sgrif]

Same with this one, would have expected it to compile: http://is.gd/RyFIEl

[sgrif]

Looks like there's some quirks in determining overlap when associated types are involved. This compiles: http://is.gd/JBPzIX, while this effectively identical code doesn't: http://is.gd/0ksLPX

[SergioBenitez]

Here's a piece of code I expected to compile with specialization:

http://is.gd/3BNbfK

#![feature(specialization)]

use std::str::FromStr;

struct Error;

trait Simple<'a> {
    fn do_something(s: &'a str) -> Result<Self, Error>;
}

impl<'a> Simple<'a> for &'a str {
     fn do_something(s: &'a str) -> Result<Self, Error> {
        Ok(s)
    }
}

impl<'a, T: FromStr> Simple<'a> for T {
    fn do_something(s: &'a str) -> Result<Self, Error> {
        T::from_str(s).map_err(|_| Error)
    }
}

fn main() {
    // Do nothing. Just type check.
}

Compilation fails with the compiler citing implementation conflicts. Note that &str doesn't implement FromStr, so there shouldn't be a conflict.

[aturon]

@sgrif

I had time to look at the first two examples. Here are my notes.

Example 1

First case, you have:

• FromSqlRow<ST, DB> for T where T: FromSql<ST, DB>
• FromSqlRow<(ST, SU), DB> for (T, U) where T: FromSqlRow<ST, DB>, U: FromSqlRow<SU, DB>,

The problem is that these impls overlap but neither is more specific than the other:

• You can potentially have a T: FromSql<ST, DB> where T is not a pair (so it matches the first impl but not the second).
• You can potentially have a (T, U) where:
  • T: FromSqlRow<ST, DB>,
  • U: FromSqlRow<SU, DB>, but not
  • (T, U): FromSql<(ST, SU), DB>
  • (so the second impl matches, but not the first)
• The two impls overlap because you can have a (T, U) such that:
  • T: FromSqlRow<ST, DB>
  • U: FromSqlRow<SU, DB>
  • (T, U): FromSql<(ST, SU), DB>

This is the kind of situation that lattice impls would allow -- you'd have to write a third impl for the overlapping case, and say what it should do. Alternatively, negative trait impls might give you a way to rule out overlap or otherwise tweak which matches are possible.

Example 2

You have:

• Queryable<ST, DB> for T where T: FromSqlRow<ST, DB>
• Queryable<Nullable<ST>, DB> for Option<T> where T: Queryable<ST, DB>

These overlap because you can have Option<T> where:

• T: Queryable<ST, DB>
• Option<T>: FromSqlRow<Nullable<ST>, DB>

But neither impl is more specific:

• You can have a T such that T: FromSqlRow<ST, DB> but T is not an Option<U> (matches first impl but not second)
• You can have an Option<T> such that T: Queryable<ST, DB> but not Option<T>: FromSqlRow<Nullable<ST>, DB>

[aturon]

@SergioBenitez

Compilation fails with the compiler citing implementation conflicts. Note that &str doesn't implement FromStr, so there shouldn't be a conflict.

The problem is that the compiler is conservatively assuming that &str might come to implement FromStr in the future. That may seem silly for this example, but in general, we add new impls all the time, and we want to protect downstream code from breaking when we add those impls.

This is a conservative choice, and is something we might want to relax over time. You can get the background here:

• http://smallcultfollowing.com/babysteps/blog/2015/01/14/little-orphan-impls/
• rust-lang/rfcs#1023
• rust-lang/rfcs#1053
• rust-lang/rfcs#1148

[sgrif]

Thank you for clarifying those two cases. It makes complete sense now

On Tue, Mar 22, 2016, 6:34 PM Aaron Turon notifications@github.com wrote:

@SergioBenitez https://github.com/SergioBenitez

Compilation fails with the compiler citing implementation conflicts. Note
that &str doesn't implement FromStr, so there shouldn't be a conflict.

The problem is that the compiler is conservatively assuming that &str
might come to implement FromStr in the future. That may seem silly for
this example, but in general, we add new impls all the time, and we want to
protect downstream code from breaking when we add those impls.

This is a conservative choice, and is something we might want to relax
over time. You can get the background here:

http://smallcultfollowing.com/babysteps/blog/2015/01/14/little-orphan-impls/

• rust-lang/rfcs#1023 rust-lang/rfcs#1023
• rust-lang/rfcs#1053 rust-lang/rfcs#1053
• rust-lang/rfcs#1148 rust-lang/rfcs#1148

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#31844 (comment)

[SergioBenitez]

@aturon

The problem is that the compiler is conservatively assuming that &str might come to implement FromStr in the future. That may seem silly for this example, but in general, we add new impls all the time, and we want to protect downstream code from breaking when we add those impls.

Isn't this exactly what specialization is trying to address? With specialization, I would expect that even if an implementation of FromStr for &str were added in the future, the direct implementation of the Simple trait for &str would take precedence.

[sgrif]

@SergioBenitez you need to put default fn in the more general impl. Your
example isn't specializable.

On Tue, Mar 22, 2016, 6:54 PM Sergio Benitez notifications@github.com
wrote:

@aturon https://github.com/aturon

The problem is that the compiler is conservatively assuming that &str
might come to implement FromStr in the future. That may seem silly for this
example, but in general, we add new impls all the time, and we want to
protect downstream code from breaking when we add those impls.

Isn't this exactly what specialization is trying to address? With
specialization, I would expect that even if an implementation of FromStr
for &str were added in the future, the direct implementation for the
trait for &str would take precedence.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#31844 (comment)

[burdges]

I think "default" trait items being automatically considered default sounds confusing. You might want both parametricity for a trait like in Haskell, etc. along side with easing the impls. Also you cannot easily grep for them like you can for default. It's not hard to both type the default keyword and give a default implementation, but they cannot be separated as is. Also, if one wants to clarify the language, then these "default" trait items could be renamed to "trait proposed" items in documentation.

[Stebalien]

Note from #32999 (comment): if we do go with the lattice rule (or allow negative constraints), the "use an intermediate trait" trick to prevent further specialization of something will no longer work.

[arielb1]

@Stebalien

Why won't it work? The trick limits the specialization to a private trait. You can't specialize the private trait if you can't access it.

[Stebalien]

@arielb1 Ah. Good point. In my case, the trait isn't private.

[arielb1]

I don't think the "externals can't specialize because orphan forward-compatibility + coherence rulea" reasoning is particularly interesting or useful. Especially when we don't commit to our specific coherence rules.

[burdges]

Is there a way to access an overridden default impl? If so, this could aid in constructing tests. See Design By Contract and libhoare.

[rphmeier]

Allowing projection of default associated types during type-checking will allow enforcing type inequality at compile-time: https://gist.github.com/7c081574958d22f89d434a97b626b1e4

#![feature(specialization)]

pub trait NotSame {}

pub struct True;
pub struct False;

pub trait Sameness {
    type Same;
}

mod internal {
    pub trait PrivSameness {
        type Same;
    }
}

use internal::PrivSameness;

impl<A, B> Sameness for (A, B) {
    type Same = <Self as PrivSameness>::Same;
}

impl<A, B> PrivSameness for (A, B) {
    default type Same = False;
}
impl<A> PrivSameness for (A, A) {
    type Same = True;
}

impl<A, B> NotSame for (A, B) where (A, B): Sameness<Same=False> {}

fn not_same<A, B>() where (A, B): NotSame {}

fn main() {
    // would compile
    not_same::<i32, f32>();

    // would not compile
    // not_same::<i32, i32>();
}

edited per @burdges' comment

[burdges]

Just fyi @rphmeier one should probably avoid is.gd because it does not resolve for Tor users due to using CloudFlare. GitHub works fine with full URLs. And play.rust-lang.org works fine over Tor.

[SimonSapin]

@burdges FWIW play.rust-lang.org itself uses is.gd for its "Shorten" button.

It can probably be changed, though: https://github.com/rust-lang/rust-playpen/blob/9777ef59b/static/web.js#L333

[TeXitoi]

@Boiethios Your usecase is working if you default on the fn and not on the impl:

#![feature(specialization)]

trait Count {
    fn count(self) -> usize;
}

impl<T> Count for T {
    default fn count(self) -> usize {
        1
    }
}

impl<T> Count for T
where
    T: IntoIterator,
    T::Item: Count,
{
    fn count(self) -> usize {
        let i = self.into_iter();

        i.map(|x| x.count()).sum()
    }
}

fn main() {
    let v = vec![1, 2, 3];
    assert_eq!(v.count(), 3);

    let v = vec![vec![1, 2, 3], vec![4, 5, 6]];
    assert_eq!(v.count(), 6);
}

[alexreg]

Has the soundness hole still not been fixed yet?

[mark-i-m]

@alexreg I don't think so. See http://smallcultfollowing.com/babysteps/blog/2018/02/09/maximally-minimal-specialization-always-applicable-impls/

My guess is that everyone's focused on the edition right now...

[alexreg]

Okay thanks... seems like this issue is dragging on forever, but fair enough. It's tough, I know. And attention is directed elsewhere right now unfortunately.

[pythonesque]

Can someone more concretely explain the rationale behind not allowing projections for default associated types in fully-monomorphic cases? I have a use case where I would like that functionality (in particular, it would be semantically incorrect for the trait to ever be invoked with types that weren't fully monomorphic), and if there's no soundness issue I don't completely understand why it's disallowed.

[sgrif]

@pythonesque There's some discussion at #42411

[pythonesque]

Ah, I understand if it turns out that projection interacts badly with specialization in general. . And it is indeed true that what I want is of a "negative reasoning" flavor (though closed traits would not really be sufficient).

Unfortunately, I'm not sure if there's really any way to do what I want without such a feature: I'd like to have an associated type that outputs "True" when two passed-in types implementing a particular trait are syntactically equal, and "False" when they aren't (with the "False" case triggering a more expensive trait search which can decide whether they are "semantically" equal). The only real alternative seems (to me) to be to just always do the expensive search; which is fine in theory, but it can be a lot more expensive.

(I could work around this if the trait were intended to be closed, by just enumerating every possible pair of constructors in the head position and having them output True or False; but it's intended to be open to extension outside the repository, so that can't possibly work, especially since implementations in two different user repositories wouldn't necessarily know about each other).

Anyway, maybe this is just an indication that what I want to do is a bad fit for the trait system and I should switch to some other mechanism, like macros :P

[gnzlbg]

And it is indeed true that what I want is of a "negative reasoning" flavor (though closed traits would not really be sufficient).

An alternative to negative reasoning is requiring that a type implements only one trait of a closed set of traits, such that implementations with other other traits in the set cannot overlap (e.g. T implements one of { Float | Int | Bool | Ptr }).

[pythonesque]

Even if there were a way to enforce that in Rust (which there isn't, AFAIK?), I do not think that would solve my problem. I would like users in different crates to be able to implement an arbitrary number of new constants, which should compare equal only to themselves and unequal to every other defined constant, including ones unknown at crate definition time. I don't see how any closed set of traits (or even set of families of traits) can accomplish that goal by itself: this is a problem that fundamentally can't be solved without looking directly at the types. The reason it would be workable with default projections is that you could default everything to "don't compare equal" and then implement equality of your new constant to itself in whatever crate you defined the constant in, which wouldn't run afoul of orphan rules because all the types in the trait implementation were in the same crate. If I wanted almost any such rule but equality, even this wouldn't work, but equality is good enough for me :)

[rmanoka]

On present nightly, this works:

trait Foo {}
trait Bar {}

impl<T: Bar> Foo for T {}
impl Foo for () {}

but even with specialization, and using nightly, this does not:

#![feature(specialization)]

trait Foo<F> {}
trait Bar<F> {}

default impl<F, T: Bar<F>> Foo<F> for T {}
impl<F> Foo<F> for () {}

Does this have a rationale or is it a bug?

[Boscop]

@rmanoka Isn't this just the normal orphan rules? In the first case, no downstream crate could impl Bar for () so the compiler allows this, but in the second example, a downstream crate could impl Bar<CustomType> for () which would conflict with your default impl.

[rmanoka]

@Boscop In that scenario, the default impl should anyway be overridden by the non-default one below. For instance, if I had: impl Bar<bool> for () {} added before the other impls, then I would expect it to work (as per RFC / expectation). Isn't that correct?

Digging deeper along the lines of the counter-example you've mentioned, I realise (or believe) that the example satisfies the "always-applicable" test, and may be being worked on.

[zserik]

This issue probably depends on #45814.