Tracking issue for specialization (RFC 1210)

[nikomatsakis]

This is a tracking issue for specialization (rust-lang/rfcs#1210).

Major implementation steps:

• Land Implement RFC 1210: impl specialization #30652 =)
• Restrictions around lifetime dispatch (currently a soundness hole)
• default impl (support default impl for specialization #37653)
• Integration with associated consts
• Bounds not always properly enforced (Trait bounds not checked on specializable associated types #33017)
• Should we permit empty impls if parent has no default members? 🛠️ specialization permits empty impls when parent has no default items #48444
• implement "always applicable" impls 🔬 implement "always applicable impls" #48538
• describe and test the precise cycle conditions around creating the specialization graph (see e.g. this comment, which noted that we have some very careful logic here today)

Unresolved questions from the RFC:

• Should associated type be specializable at all?
• When should projection reveal a default type? Never during typeck? Or when monomorphic?
• Should default trait items be considered default (i.e. specializable)?
• Should we have default impl (where all items are default) or partial impl (where default is opt-in); see support default impl for specialization #37653 (comment) for some relevant examples of where default impl is limiting.
• How should we deal with lifetime dispatchability?

Note that the specialization feature as implemented currently is unsound, which means that it can cause Undefined Behavior without unsafe code. min_specialization avoids most of the pitfalls.

[aturon]

Some additional open questions:

• Should we revisit the orphan rules in the light of specialization? Are there ways to make things more flexible now?
• Should we extend the "chain rule" in the RFC to something more expressive, like the so-called "lattice rule"?
• Related to both of the above, how does negative reasoning fit into the story? Can we recover the negative reasoning we need by a clever enough use of specialization/orphan rules, or should we make it more first-class?

[arielb1]

I am not sure that specialization changes the orphan rules:

• The "linking" orphan rules must stay the same, because otherwise you would not have safe linking.
• I don't think the "future compatibility" orphan rules should change. Adding a non-specializable impl under you would still be a breaking change.

Worse than that, the "future compatibility" orphan rules keep cross-crate specialization under pretty heavy control. Without them, default-impls leaving their methods open becomes much worse.

I never liked explicit negative reasoning. I think the total negative reasoning specialization provides is a nice compromise.

[sgrif]

Should this impl be allowed with specialization as implemented? Or am I missing something?
http://is.gd/3Ul0pe

[sgrif]

Same with this one, would have expected it to compile: http://is.gd/RyFIEl

[sgrif]

Looks like there's some quirks in determining overlap when associated types are involved. This compiles: http://is.gd/JBPzIX, while this effectively identical code doesn't: http://is.gd/0ksLPX

[SergioBenitez]

Here's a piece of code I expected to compile with specialization:

http://is.gd/3BNbfK

#![feature(specialization)]

use std::str::FromStr;

struct Error;

trait Simple<'a> {
    fn do_something(s: &'a str) -> Result<Self, Error>;
}

impl<'a> Simple<'a> for &'a str {
     fn do_something(s: &'a str) -> Result<Self, Error> {
        Ok(s)
    }
}

impl<'a, T: FromStr> Simple<'a> for T {
    fn do_something(s: &'a str) -> Result<Self, Error> {
        T::from_str(s).map_err(|_| Error)
    }
}

fn main() {
    // Do nothing. Just type check.
}

Compilation fails with the compiler citing implementation conflicts. Note that &str doesn't implement FromStr, so there shouldn't be a conflict.

[aturon]

@sgrif

I had time to look at the first two examples. Here are my notes.

Example 1

First case, you have:

• FromSqlRow<ST, DB> for T where T: FromSql<ST, DB>
• FromSqlRow<(ST, SU), DB> for (T, U) where T: FromSqlRow<ST, DB>, U: FromSqlRow<SU, DB>,

The problem is that these impls overlap but neither is more specific than the other:

• You can potentially have a T: FromSql<ST, DB> where T is not a pair (so it matches the first impl but not the second).
• You can potentially have a (T, U) where:
  • T: FromSqlRow<ST, DB>,
  • U: FromSqlRow<SU, DB>, but not
  • (T, U): FromSql<(ST, SU), DB>
  • (so the second impl matches, but not the first)
• The two impls overlap because you can have a (T, U) such that:
  • T: FromSqlRow<ST, DB>
  • U: FromSqlRow<SU, DB>
  • (T, U): FromSql<(ST, SU), DB>

This is the kind of situation that lattice impls would allow -- you'd have to write a third impl for the overlapping case, and say what it should do. Alternatively, negative trait impls might give you a way to rule out overlap or otherwise tweak which matches are possible.

Example 2

You have:

• Queryable<ST, DB> for T where T: FromSqlRow<ST, DB>
• Queryable<Nullable<ST>, DB> for Option<T> where T: Queryable<ST, DB>

These overlap because you can have Option<T> where:

• T: Queryable<ST, DB>
• Option<T>: FromSqlRow<Nullable<ST>, DB>

But neither impl is more specific:

• You can have a T such that T: FromSqlRow<ST, DB> but T is not an Option<U> (matches first impl but not second)
• You can have an Option<T> such that T: Queryable<ST, DB> but not Option<T>: FromSqlRow<Nullable<ST>, DB>

[aturon]

@SergioBenitez

Compilation fails with the compiler citing implementation conflicts. Note that &str doesn't implement FromStr, so there shouldn't be a conflict.

The problem is that the compiler is conservatively assuming that &str might come to implement FromStr in the future. That may seem silly for this example, but in general, we add new impls all the time, and we want to protect downstream code from breaking when we add those impls.

This is a conservative choice, and is something we might want to relax over time. You can get the background here:

• http://smallcultfollowing.com/babysteps/blog/2015/01/14/little-orphan-impls/
• Rebalancing coherence. rfcs#1023
• More flexible coherence rules that permit overlap rfcs#1053
• Mutually exclusive traits rfcs#1148

[sgrif]

Thank you for clarifying those two cases. It makes complete sense now

On Tue, Mar 22, 2016, 6:34 PM Aaron Turon notifications@github.com wrote:

@SergioBenitez https://github.com/SergioBenitez

Compilation fails with the compiler citing implementation conflicts. Note
that &str doesn't implement FromStr, so there shouldn't be a conflict.

The problem is that the compiler is conservatively assuming that &str
might come to implement FromStr in the future. That may seem silly for
this example, but in general, we add new impls all the time, and we want to
protect downstream code from breaking when we add those impls.

This is a conservative choice, and is something we might want to relax
over time. You can get the background here:

http://smallcultfollowing.com/babysteps/blog/2015/01/14/little-orphan-impls/

• Rebalancing coherence. rfcs#1023 Rebalancing coherence. rfcs#1023
• More flexible coherence rules that permit overlap rfcs#1053 More flexible coherence rules that permit overlap rfcs#1053
• Mutually exclusive traits rfcs#1148 Mutually exclusive traits rfcs#1148

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#31844 (comment)

[SergioBenitez]

@aturon

The problem is that the compiler is conservatively assuming that &str might come to implement FromStr in the future. That may seem silly for this example, but in general, we add new impls all the time, and we want to protect downstream code from breaking when we add those impls.

Isn't this exactly what specialization is trying to address? With specialization, I would expect that even if an implementation of FromStr for &str were added in the future, the direct implementation of the Simple trait for &str would take precedence.

[sgrif]

@SergioBenitez you need to put default fn in the more general impl. Your
example isn't specializable.

On Tue, Mar 22, 2016, 6:54 PM Sergio Benitez notifications@github.com
wrote:

@aturon https://github.com/aturon

The problem is that the compiler is conservatively assuming that &str
might come to implement FromStr in the future. That may seem silly for this
example, but in general, we add new impls all the time, and we want to
protect downstream code from breaking when we add those impls.

Isn't this exactly what specialization is trying to address? With
specialization, I would expect that even if an implementation of FromStr
for &str were added in the future, the direct implementation for the
trait for &str would take precedence.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#31844 (comment)

[burdges]

I think "default" trait items being automatically considered default sounds confusing. You might want both parametricity for a trait like in Haskell, etc. along side with easing the impls. Also you cannot easily grep for them like you can for default. It's not hard to both type the default keyword and give a default implementation, but they cannot be separated as is. Also, if one wants to clarify the language, then these "default" trait items could be renamed to "trait proposed" items in documentation.

[Stebalien]

Note from #32999 (comment): if we do go with the lattice rule (or allow negative constraints), the "use an intermediate trait" trick to prevent further specialization of something will no longer work.

[arielb1]

@Stebalien

Why won't it work? The trick limits the specialization to a private trait. You can't specialize the private trait if you can't access it.

[Stebalien]

@arielb1 Ah. Good point. In my case, the trait isn't private.

[arielb1]

I don't think the "externals can't specialize because orphan forward-compatibility + coherence rulea" reasoning is particularly interesting or useful. Especially when we don't commit to our specific coherence rules.

[burdges]

Is there a way to access an overridden default impl? If so, this could aid in constructing tests. See Design By Contract and libhoare.

[rphmeier]

Allowing projection of default associated types during type-checking will allow enforcing type inequality at compile-time: https://gist.github.com/7c081574958d22f89d434a97b626b1e4

#![feature(specialization)]

pub trait NotSame {}

pub struct True;
pub struct False;

pub trait Sameness {
    type Same;
}

mod internal {
    pub trait PrivSameness {
        type Same;
    }
}

use internal::PrivSameness;

impl<A, B> Sameness for (A, B) {
    type Same = <Self as PrivSameness>::Same;
}

impl<A, B> PrivSameness for (A, B) {
    default type Same = False;
}
impl<A> PrivSameness for (A, A) {
    type Same = True;
}

impl<A, B> NotSame for (A, B) where (A, B): Sameness<Same=False> {}

fn not_same<A, B>() where (A, B): NotSame {}

fn main() {
    // would compile
    not_same::<i32, f32>();

    // would not compile
    // not_same::<i32, i32>();
}

edited per @burdges' comment

[burdges]

Just fyi @rphmeier one should probably avoid is.gd because it does not resolve for Tor users due to using CloudFlare. GitHub works fine with full URLs. And play.rust-lang.org works fine over Tor.

[SimonSapin]

@burdges FWIW play.rust-lang.org itself uses is.gd for its "Shorten" button.

It can probably be changed, though: https://github.com/rust-lang/rust-playpen/blob/9777ef59b/static/web.js#L333

[joshlf]

Is there any documentation on what's included in min_specialization? Does anyone have a sense of the blockers for stabilization?

[the8472]

I don't think there's much documentation because nobody is working on stabilizing it.

min_specialization covers

1. the default keyword on fn in impls
2. specializing concrete type/const generic values where the base is a strict superset.
3. some weird rules about how generic args can and can't be repeated in specializations (parts of the "always applicable" logic)

It doesn't include trait specialization or associated item specialization. It also doesn't include a good way to handle partially overlapping cases. In a few cases one can build hierarchies of multiple specializations, but not always.

To specialize on traits instead of concrete types you'll additionally need the rustc_attrs feature to get

• #[rustc_specialization_trait]
• #[rustc_unsafe_specialization_marker]

You can grep through the standard library looking for default fn to see some uses of specialization. Afaik nobody is working on stabilizing it and the rustc_ traits aren't meant for stabilization in the first place, so there's no documentation for downstream use. The std dev guide has a page though: https://std-dev-guide.rust-lang.org/policy/specialization.html

[mark-i-m]

I believe either Niko or Aaron Turon wrote a blog post about min specialization a fee years ago.

[vacuus]

https://aturon.github.io/blog/2017/07/08/lifetime-dispatch/
https://internals.rust-lang.org/t/shipping-specialization-a-story-of-soundness/5507
https://smallcultfollowing.com/babysteps/blog/2018/02/09/maximally-minimal-specialization-always-applicable-impls/
https://internals.rust-lang.org/t/blog-post-maximally-minimal-specialization-always-applicable-impls/6739
Perhaps the trait system rework will spur this on?

[the8472]

The specialization feature is marked unstable for that reason (as also explained in these blog posts #31844 (comment)), among others.

The compiler warns:

warning: the feature specialization is incomplete and may not be safe to use and/or cause compiler crashes
= note: see issue #31844 #31844 for more information
= help: consider using min_specialization instead, which is more stable and complete
= note: #[warn(incomplete_features)] on by default

So, for now use min_specialization and the specialization traits. #31844 (comment)
Those are more usable.

[oli-obk]

Tracking issues have a tendency to become unmanageable. Please open a dedicated new issue and label it with F-specialization `#![feature(specialization)]` for absolutely any topics you want to discuss or have questions about. See rust-lang/compiler-team#739 for details and discussions on this prospective policy.