RFC: inherent trait implementation

[newpavlov]

Continuation of #2309.

Fixes: #1880,#1971

This RFC allows us to write the following code:

#[inherent]
impl Bar for Foo {
    // code
}

Which allows methods from Bar trait to be used on Foo instances without having Bar in the scope.

Rendered

[newpavlov]

cc @nikomatsakis, @aturon

This RFC assumes that rust-lang/rust#48444 will be treated as a feature and not as a bug.

[aturon]

To clarify: #2309 was postponed until we figure out the story for delegation. While you talk about it a bit in this new RFC, it's not clear what in this proposal changes the situation compared to when the lang team previous took up this question. Can you expand on your thinking here?

[newpavlov]

I was under impression that the previous RFC was closed due to the lack of reaction from @Diggsey to add requested changes, and not postponed. (ctrl+f "postpone" yields zero results)

While I agree that delegation RFC and inherent traits should be discussed together, in my opinion RFCs have orthogonal scopes and similar only in the end effect. As was shown in the text, delegation can be nicely composed with #[inhrent] attribute and there is no need to overload delegation with an additional functionality.

Yes, the new delegation RFC draft mentions inherent trait impls as a possible future extension, but I don't think that using delegate is a correct approach here. Also in my opinion inherent trait implementations can have a much bigger impact on the ecosystem (including stdlib and core) than the delegation RFC, thus it should consider as one of the main topics and not as a vague future extension.

[Diggsey]

@newpavlov I don't think that's an accurate assessment. The RFC was closed because:

We discussed this in the lang team meeting today, and everyone agreed it would be nice to provide "inherent trait methods" of some sort. However, we also felt that this feature would probably fall out naturally from a solution to the more general problem of trait and method delegation. We'd like to see another RFC for delegation before we accept a feature like this.

ie. the lang team wanted a more general solution

[newpavlov]

@Diggsey
Yeah, my mistake. Though by reading discussion of the new version of delegation RFC it looks like this use case is not going to "fall out naturally" any time soon.

@aturon
So should I close this RFC until lang-team decides how inherent traits and delegation will interact with each other or should I leave it to be for now?

[aturon]

@newpavlov Sorry for the delay, was out on vacation.

Yes, I think it should probably be closed for the time being, especially since there's now a delegation RFC. Would be good to revisit after Rust 2018 ships!

[dwijnand]

Where's the delegation RFC? My searches failed to find anything.

[newpavlov]

@dwijnand #2393

[newpavlov]

@aturon
How about reopening this PR?

[Manishearth]

I think this should be explicitly specced out, especially since this expansion actually introduces an ambiguity in method calls.

[Manishearth]

This should be fleshed out, including:

• What impl blocks it can be applied to (can the trait be foreign?)
• Whether it works with #[fundamental]
• Does it error if you have an inherent method with the same name?

[burdges]

I'd think it should error if you have an inherent method with the same name, like if you had two inherent methods with the same name.

[newpavlov]

What impl blocks it can be applied to (can the trait be foreign?)

Yes, it can. One of the main use-cases for inherent trait implementations is implementation of crates defined in external crates.

Whether it works with #[fundamental]

I am not sure why it should not, though I can't say I fully understand #[fundamental] semantics.

Does it error if you have an inherent method with the same name?

Initially I though it should error, but as I wrote below probably it may be better to issue a warning and shadow inherent trait methods by true inherent methods.

I will try to update the text based on your review! Thanks!

[Manishearth]

If this works the way I think it does, this actually changes our trait stability story, and that's a major drawback.

Currently, adding a defaulted method to a trait is not a major breaking change, at worst it will cause ambiguity errors, so it's considered minor.

Now, a dependency can add a method to a trait you #[inherent] impl, which can clash with a method of your own, causing your build to fail in a way that requires you to change your API to fix. We're largely okay with builds failing due to new ambiguities (clashing method names across traits, adding something to a module that's glob imported, etc) and such things are categorized as minor, which basically means it's fine to do as long as the fallout isn't too much. With this RFC, adding a defaulted method has the potential to break a library user in a way that requires them to rename a method in their public API, causing a breaking change for them.

This should be explored and addressed in this RFC, and as a bare minimum should be called out in this section.

(One "fix" is to only allow local traits)

[Havvy]

Another fix is to list out the method names that are inherent, although that becomes quite a burden for e.g. iterator methods.

[burdges]

I'm unsure if the existing situation should be classified as so minor because you interact with many types almost exclusively through their traits. As an example, there are definitely traits with a new() method that defies the convention of being inherent new(), so those traits adding new() created exactly the same breakage you describe here.

As a fix, I'd suggest #[inherent] being a method attribute for items inside an impl, so

impl TraitWIthBadMethodNames for MyType {
    fn new() -> Self { .. }  // Not inherent

    #[inherent]
    fn new_plus() -> Self;  // Inherent but default body used

    #[inherent]
    fn foo() { .. }  // Inherent with body supplied here
}

And #[inherent] applied to the impl is equivalent to it being applied to all methods and associated types and constants.

[Manishearth]

As an example, there are definitely traits with a new() method that defies the convention of being inherent new(), so those traits adding new() created exactly the same breakage you describe here.

Yes, but this breakage is easily fixed by using UFCS. This is not true for breakage caused in the world of #[inherent].

The "minor" terminology comes from the API evolution RFC, such changes may cause crates to stop compiling, however:

• This can be fixed with a trivial change local to the crate
• The fix does not break upstream crates

Without this notion of "minor" being allowed, crates wouldn't be able to add anything new (types, traits, functions, or methods) without it being considered a breaking change.

Furthermore, the API evolution RFC mentions in the trait method case that you should check to ensure the fallout isn't too great; which is where your new() example falls short: a trait adding a new() method would probably have lots of fallout.

In other words, when I use the term "minor" here, I'm using a precisely defined term from another RFC. Whether or not it is actually "minor" is irrelevant, I'm talking about what we do and don't consider breaking, which we have specced in terms of this major/minor categorization.

[newpavlov]

Will it be possible to implement shadowing of inherent trait methods by true inherent methods? Compiler will warn on clashing inherent names while building crate which uses #[inherent], but will use true inherent method by default and for trait method you will have to use explicit Trait::foo(value). This way we will avoid code breakage on "minor" upstream changes.

Though I think in practice such collisions should be extremely rare.

[joshtriplett]

Will it be possible to implement shadowing of inherent trait methods by true inherent methods?

I think that would be the ideal fix here: an inherent trait's methods get shadowed by the type's own methods. They could produce a warning lint when compiling the crate itself, so that people notice, and then cap-lints will suppress that when compiling a dependency.

[burdges]

How are polymorphic traits handled? I'd presume

#[inherent]
impl<'de> Deserialize<'de> for MyType { .. }

works roughly like

impl MyType {
    fn deserialize<'de,D>(deserializer: D) -> Result<Self, D::Error>
    where D: Deserializer<'de> { ... }
}

Yet, how should separate impl blocks be handled?

#[inherent]
impl Borrow<str> for MyType { }

#[inherent]
impl Borrow<[u8]> for MyType { }

We'd likely want roughly

impl MyType {
    fn borrow<R>(&self) -> R
    where Self: Borrow<R>
    { self.borrow() }
}

[newpavlov]

@burdges
I think that initial implementation should simply forbid inherent implementations for polymorphic traits. Later we could do an extension, though I don't see how it can be done nicely right now.

[fbstj]

is there a reason (that I'm not seeing) that this cannot be done with a macro that just implements the wrapping functions in a generated impl block?

[newpavlov]

It can be done, but will result in code and documentation bloat. Also use of procedural macros will increase compilation times.

[burdges]

fn aliases would provide a nice solution here too. I suppose fn aliases should wait until delegation gets sorted out, although argument for waiting is less compelling than for this RFC's proposal.

[Michael-F-Bryan]

Does this need to be an RFC? As far as I'm aware, it could be implemented using a normal procedural macro and uploaded to crates.io. That also gives us a place to iterate on the implementation before anything is merged.

Kinda like how we used the work from failure to improve std::error::Error's API.

[joshtriplett]

Following up on this: I went through previous feedback, resolved a couple of threads, and commented in others. I think this is now ready to update to incorporate the remaining feedback.

[QuineDot]

CC @petrochenkov, who has expressed concern over complicating method resolution further without an overarching vision.

[QuineDot]

In addition to the concerns about privacy, bounds on the trait methods should be considered in the reference-level explanation.

Example:

trait Foo {
    fn f(&self) { }
    fn g(&self) where Self: Clone {}
    fn h(self) where Self: Sized {}
}

struct NotCloneAndNotAlwaysSized<T: ?Sized>(T);
impl Foo for NotCloneAndNotAlwaysSized<str> {}
impl NotCloneAndNotAlwaysSized<str> {
    pub fn f(&self) { <Self as Foo>::f(self) }
    pub fn g(&self) where Self: Clone { <Self as Foo>::g(self) }    
    pub fn h(self) where Self: Sized { <Self as Foo>::h(self) }
}

As the error says, trivial bounds can resolve the failures, so this may not be a huge deal if/when those are stabilized.

If the bounds aren't mirrored on the inherent methods, they (correctly) cannot compile (as the trait method cannot be called). Alternatively, I suppose, the inherent methods could be not-generated if the bounds don't match.

[QuineDot]

I would also appreciate some consideration of the interaction with dyn Trait specifically. Namely, today trait methods of Trait are in some sense treated like inherent methods of dyn Trait. They are not actually inherent methods, as you can still supply your own non-conflicting method with the same name. However, you can never call your inherent dyn Trait method that shadows a trait method of Trait. This is true even if the trait method has bounds making the trait method uncallable on dyn Trait (e.g. Self: Sized).

As far as the contributors in the linked issue (and dupes) seem to think, however, this could be changed backwards-compatibly. I'm in favor of that, as it reduces complexity and increases expressiveness.

As relates to this RFC, if we gain another level of precedence (explicit inherent methods over #[inherent] methods over trait methods), it seems like this could all be consistently resolved:

• dyn Trait has a compiler-supplied implementation of Trait
• The implementation conceptually uses #[inherent], which is why the trait is implicitly in scope
• The door is still open to let explicit inherent methods to take precedence over the now-#[inherent] method on dyn Trait

(But if I'm missing something, I'd love to hear about it.)

Where as if we end up in a world where explicit inherent methods conflict with #[inherent] methods (as implied by the current reference-level explanation), I'm concerned that it would still be too tempting to retcon dyn Trait's special method resolution as an #[inherent] implementation, as that would close the door on allowing explicit inherent methods to take precedence.

I would rather the door not be closed on #51402 en passant, without proper consideration.

This can be considered one piece of the desired overarching method resolution vision.

[KatsuoRyuu]

In short how i see all this is similar to method overloads/overrides in other languages (im not a fan it creates a mess of a code structure)

If this is really something that will go in my suggestion is #[inherent] should be the lowest priority and anything that wants should be able to override it both in param and return value. But this would create a huge mess with the compiler and increase the complexity and understanding of the code too.

So why do i say that?

#[derive(Clone)]
pub struct Foo {
    x: i32,
    y: i32,
}


trait Multiply {
    fn multiply(&self) -> i32;
}

trait Multiple {
    fn multiply(&self, multiplier: i32) -> Vec<Self> where Self: Sized;
}


impl Multiple for Foo {
    fn multiply(&self, multiplier: i32) -> Vec<Self> where Self: Sized {
        let mut v = Vec::new();
        for i in 0..multiplier{
            v.push(self.clone())
        }
        v
    }
}

impl Multiply for Foo {
    fn multiply(&self) -> i32 {
        self.x * self.y
    }
}

Now in this situation the functions are not at all the same but having the same name, In normal circumstances i can operate out from the traits or depending on the modules needs i can use just the trait i need no complications there, Adding #[inherent] would complicate this for many libraries as well as projects - while solving other problems.

To work with this, any direct implemntations should take presedence over #[inherent], any dyn trait/Trait should have presedence over it as well.

This is the end will hide a large amount of code underneath the surface and you will be unaware, unless you read everything in the libraries you use what functionality you have.

Imagine someone hides some malisous code somewhere in a file that has #[inherent] normally i can follow my trait and figure out whats implemented, now with #[inherent] i no longer know if a custom trait has been defined and implemented in some third party library, and i will be using it without my knowledge.

This type of "magic" makes it hard to track whats going on.

So to resolve this, #[inherent] should also only be allowd to be used with types/traits from the current package and not with types in other packages.

Inserting this restriction will reduce the usability significantly.

So i dont see it that usable, unless you give declaration of need, but then you are back to be using traits.

[moulins]

If this works the way I think it does, this actually changes our trait stability story, and that's a major drawback.

Currently, adding a defaulted method to a trait is not a major breaking change, at worst it will cause ambiguity errors, so it's considered minor.

Now, a dependency can add a method to a trait you #[inherent] impl, which can clash with a method of your own, causing your build to fail in a way that requires you to change your API to fix. We're largely okay with builds failing due to new ambiguities (clashing method names across traits, adding something to a module that's glob imported, etc) and such things are categorized as minor, which basically means it's fine to do as long as the fallout isn't too much. With this RFC, adding a defaulted method has the potential to break a library user in a way that requires them to rename a method in their public API, causing a breaking change for them.

This should be explored and addressed in this RFC, and as a bare minimum should be called out in this section.

(One "fix" is to only allow local traits)

I haven't seen this proposed anywhere in the thread, but I believe a solution to this would be to allow #[inherent] to apply to individual trait methods; then, for non-local traits:

• #[inherent] impl Trait for Foo { ... } would be disallowed, resolving the semver concern;
• impl Trait for Foo { #[inherent] fn foo() { ... } } would still be allowed; there is no concern here as any new default methods in Trait won't be treated as #[inherent].

[burdges]

We clearly want inherent non-local traits, and inherent trait methods with default bodies, which breaks impl Trait .. { #[inheret] fn .. }. I think @cramertj's comment completely resolves shadowing concerns: Inherent impls always beat inherent trait methods in name resolution.

#[inherent] impl LocalTrait for ForeignType should maybe be forbidden, which also forbids say #[inherent] impl<T: Clone> ToOwned for T. We do not permit trait methods to shadow inherent methods anyways, but if someone uses such an inherent trait method then they could later be surprised. An opt-in might work somehow, ala #[inherent] impl ToOwned for MyType; or #[inherent(Trait1,Trait2,..)] pub struct ..

I still think polymorphic traits could probably be supported, at least for lifetimes.

I'm unsure if this needs privacy specifiers honestly, but if desired then we could write trait Trait { #[inherent(visibility)] fn .. to control the visibility applied by #[inherent] impl Trait for ..

We should use the inherent proc macro crate for now. I'm dubious it slows compilation as much as @newpavlov thinks.. GenericArray should cause far worse slowdowns.

[moulins]

#[inherent] impl LocalTrait for ForeignType should maybe be forbidden [...]

IMHO this should definitely be disallowed, as it'd be tantamount to defining an inherent impl block for third-party types; consider;

crate a {
  pub struct S;
}

crate b {
  pub trait Tr {
    fn f(self);
  }
  
  #[inherent]
  impl Tr for a::S {
    fn f(self) {}
  }
}

crate c {
  let s = a::S;
  // where does this come from? what if `b` is only a transitive dependency?
  s.f();
}

[joshtriplett]

We reviewed this in a @rust-lang/lang meeting today, and would like to move ahead with it.

Regarding the concern of #[inherent] impl LocalTrait for RemoteType, the proposed desugaring in the RFC would make that an error, and we do think that should be an error.

Regarding the concern of #[inherent] impl RemoteTrait for LocalType causing addition of defaulted methods to RemoteTrait being a breaking change, there is a way (albeit annoying) that the crate with that inherent impl can fix such breakage if it arises without changing their API: they could do the desugar manually and only include the methods they want. We could also (as suggested in this thread) add syntax to mark individual methods in a trait impl as #[inherent] rather than the whole impl, which would make that workaround easier. Proposal: we should add that to future work and make it an unresolved question whether we need that syntax before stabilizing.

Another option would be to augment the desugaring to say that "#[inherent] does not add an inherent forwarding method if the type has explicitly defined inherent method of the same name.". You could still get breakage that way if you had multiple #[inherent] blocks and one added a defaulted method conflicting with the other, but that's more far-fetched.

@programmerjake also made the observation that we might want the ability to have a blanket impl for a variety of types and then have a way to make that impl inherent for a specific type. Proposal: let's similarly add that as potential future work.

@rfcbot merge

@rfcbot concern decide-on-handling-for-breakage-with-new-trait-methods

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @pnkfelix
• @scottmcm
• @tmandry

Concerns:

• decide-on-handling-for-breakage-with-new-trait-methods (RFC: inherent trait implementation #2375 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[OpatrilPeter]

Apologies if I've missed something obvious -- I can see here a lot of discussion about the mechanical aspects of this feature, but, curiously, I haven't seen much about the plans for usage recommendations, despite the far reaching possibilities.

I feel that, once this feature becomes widely available, "Should I implement this trait for my type inherently or not?" will be a question for every library author, so there should be at least a broad guideline planned before then.

For example, should we say that types existing primarily for satisfying specific traits, such as std::io::BufReader for std::io::Read or std::iter::Iterator for std::iter::Map, should be the ones using it? If so, what about less obvious cases, such as, say, std::iter::IntoIterator for Vec? Where do we want to draw the line?
At extreme end, I can imagine a world where writing inherent impls becomes a recommended style where possible, making Rust feel closer to traditional OOP languages, where classes inherit from interfaces/abstract base classes.

A more concrete point on how being opinionated about usage of this feature could impact things:
If there'd be a convention that inherent impls are expected to "form a fundamental functionality" for given type, then I suspect there's a potential to effectively make the (unstable) documentation feature of "notable traits"[1] obsolete, as we could instead list inherent impls for given type.

[1] As seen, for example, in infobox for return type of Iterator::map.

[nikomatsakis]

Per the discussion on zulip, I have become convinced that it would be better to make this feature use the syntax use, like:

impl SomeType {
    pub use SomeTrait::*;  // re-export the methods for the trait implementation
}

This syntax has a few advantages:

• We can give preference to explicit method declared in the impl blocks over glob re-exports, eliminating one source of breakage (i.e., trait adds a method with a name that overlaps one of the inherent methods defined on SomeType)
• Can make just specific methods (not all of them) inherent.
• Easier to see the inherent method when scanning source.
• You can re-export with different visibility levels (e.g., pub(crate))
• It would work best if we planned to permit use SomeTrait::some_method; as a way to import methods as standalone fns, but I wish we did that.

However, in writing this, I realize an obvious disadvantage -- if the trait has more generics and things, it's not obvious how those should map. i.e., consider

struct MyType<T> { 
}

impl<T> MyType<T> {
    pub use MyTrait::foo;
}

impl<T: Debug> MyTrait for MyType<T> {
    fn foo(&self) { }
}

This would be weird -- is this an error, because the impl block says it's for all T? And what if it were trait MyTRait<X>?

[ids1024]

use syntax could also allow using things that aren't part of a trait as associated functions, associated types (rust-lang/rust#8995), etc. Which could occasionally be useful, at least.

If use syntax is extended, but doesn't support some of those things that logically seem like they should be supported, it has the downside of adding more functionality to the language with "orthogonality" issues. Similarly with the use SomeTrait::some_method example.

I like the idea, but to make it feel like a natural part of the language and not an overloading of the use keyword for something else, there are a lot of things to consider.

[RalfJung]

It would work best if we planned to permit use SomeTrait::some_method; as a way to import methods as standalone fns, but I wish we did that.

I don't entirely understand this sentence, why is there a "but" there? Or is there a negation missing somewhere?

[burdges]

I'd say drop the type parameters entirely like

impl SomeType pub(crate) use SomeTrait::{some_method};

or even

pub(crate) impl SomeType use SomeTrait::{some_method};

or

pub(crate) use SomeTrait::{some_method} impl SomeType;

[dhardy]

However, in writing this, I realize an obvious disadvantage -- if the trait has more generics and things, it's not obvious how those should map. i.e., consider

struct MyType<T> { 
}

impl<T> MyType<T> {
    pub use MyTrait::foo;
}

impl<T: Debug> MyTrait for MyType<T> {
    fn foo(&self) { }
}

Lets break this down:

• If a trait has no extra generics or bounds, no problem.
• If, as in the example above, the trait has no extra generics but its impl has extra bounds, that should be an error. The fix is easy — replicate those bounds on the "using" impl block — impl<T: Debug> MyType<T> { pub use MyTrait::foo; }.
• If the trait has extra generics in its name (e.g. From<T>), then different instantiations are different types, so for example From::<u32>::from and From::<i32>::from are not the same method. Thus, one cannot simply pub use From::from; to make it inherent. (Optionally, one could pub use From::<i32>::from; — weird, but it's a fully-qualified path to a method.)
• If a trait has an associated type, this type is known — so, provided the trait is implemented, there should be no issue re-exporting its methods and associated types: pub use Iterator::*;

---

That said, the syntax is still weird. I think I'd prefer this:

impl<T: Debug> MyType<T> {
    pub use <Self as MyTrait>::foo;
    // or the fully-qualified variant:
    pub use <MyType<T> as MyTrait>::foo;
}

[petrochenkov]

@dhardy
This almost words on nightly rustc with delegation feature enabled:

#![feature(fn_delegation)]
#![allow(incomplete_features)]

struct MyType<T> { field: T }

trait MyTrait {
    fn foo(&self) {}
    fn bar(&self) {}
    fn baz(&self) {}
}

impl<T: std::fmt::Debug> MyTrait for MyType<T> {}

impl<T: std::fmt::Debug> MyType<T> {
    pub reuse MyTrait::foo;
    pub reuse <Self as MyTrait>::bar;
    pub reuse <MyType<T> as MyTrait>::baz;
}

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=35cde317e177ea36698eb3901988a482

, except generics are not fully supported yet.

UPD: Works without generics - https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=d7d92cc51a2cd01b325f97eaaa65a01e

[CAD97]

re: concern decide-on-handling-for-breakage-with-new-trait-methods:

An alternative formulation without the breakage could be to, instead of desugaring to an additional "real" inherent implementation, have #[inherent] change name resolution for SelfTy to always act as-if the implemented trait were in scope. That is, the implementation can coexist with a "real" inherent implementation of that name, with the "real" inherent implementation shadowing the trait implementation, and causing a name resolution conflict if another trait is in scope also has items with the same name.

#[inherent] impl would still be restricted to only be allowed when a "real" inherent impl could be written, and could still warn if any of the trait associated items are shadowed by "real" inherent impls.