RFC: impl trait expressions

[canndrew]

Rendered view

This is an idea I've seen floating around for a while. I like it, so I decided to give it a proper RFC.

Summary: Add impl Trait { ... } expressions as a kind-of generalization of closure syntax.

[Centril]

The motivation feels a bit thin; it would be good to work in some real world use cases of where it would be beneficial; I'm sure you won't have any problems doing that.

[Centril]

I don't think calling it sugar does it justice; in particular, if you add type parameters to a closure, there's type inference going on to infer what the type of Self::Output is as well as what Args is.

[Centril]

Suggested change

	Which, in turn, is syntax sugar for:
	Which, in turn, is syntactic sugar for:

[Centril]

Suggested change

	With this RFC, the above code becomes syntax sugar for:
	With this RFC, the above code becomes syntactic sugar for:

[Centril]

I'd like at least the following to be discussed:

1. What is the interaction with #2229?

2. What changes are there to the grammar? and are there any ambiguities (there are, see below)

3. What happens when I implement the Fn trait and it is a subtrait once removed of FnOnce? Do I also get the supertrait impls?

4. How do I implement several traits at once? Can I do that? This ties into 3.

5. What is the type of an anonymous impl? A voldemort type?

6. Will Copy and Clone be implemented for the impl trait expression if possible like for closures?

7. What happens if I write let obj = impl Foo<'_> { ... };? Normally this would quantify a lifetime 'a so we'd have impl<'a> Foo<'a> for X { ... }...

8. What rules if any apply for turning let x = impl Foo { ... } into a trait object?

[Centril]

As for the ambiguity aforementioned, consider:

fn foo() {
    struct X;
    
    impl X {
        fn foo() {}
    };
}

This compiles today and impl X { ... } is an item followed by the empty statement ;.

According to your RFC however impl X { ... } is an expression which is made into a statement by the following ;.

This is not an insurmountable challenge but you'll need to think about how to deal with it.

[Centril]

cc @petrochenkov @eddyb @qmx ^ ideas?

[Centril]

Some prior art to consider:

• Java's anonymous classes

[clarfon]

Kotlin and C++ also have a similar feature to Java's.

[burdges]

It's too bad this never came up before -> impl Trait because this would've provided an excellent reason for the -> some Trait syntax or whatever.

[eddyb]

My version of this general idea (predating -> impl Trait IIRC):

fn main() {
    let world = "world";
    let says_hello_world = struct {
        world,
    } impl fmt::Display {
        fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
            write!(f, "hello {}", self.world)
        }
    };

    println!("{}", says_hello_world);
}

The struct {...} would be an "anonymous" struct (not a structural record, more like a closure, with its own identity, different from all other struct {...} expressions).
The advantage, IMO, is that passing the "captures" around is explicit and works for more situations (whereas closures are more limited):

struct {
    x,
    y,
} impl Default {
    fn default() -> Self {
        Self { x: 1, y: -1 }
    }
} impl Clone { // imagine `#[derive(Clone)]` generating this:
    fn clone(&self) -> Self {
        Self {
            x: self.x.clone(),
            y: self.y.clone(),
        }
    }
}

[mark-i-m]

I could be persuaded to be in favor of this, but my inclination is against it just because of Java. Anonymous classes in Java are huge source of noise and hard-to-understand code IMHO.

In addition, I would like to see the following addressed in the RFC:

• Documentation: do anonymous structs show up in rustdocs? If so, what do they look like?
• Debugging and name mangling: debugging and printing error messages with unnameable types is a pain. What does that look like in this case? Additionally, what needs to be added to a name-mangling scheme like #2603 for this to work?

[sfackler]

@mark-i-m I'd expect the answers to both of those questions to be identical to the answers for closures.

[ExpHP]

Mind that the type parameters of the Fn traits are not a stable part of Rust, so be careful not to imply that this should work

[eddyb]

This should probably be written as impl FnOnce() (cc @nikomatsakis)

[ExpHP]

Something like this which allows omitting the types of the closed over locals could be huge for macros that need to codegen implementors of a trait.

Then again, the feature might also even eliminate the need for most such macros!

[eddyb]

Aaaa I keep being reminded I need to fix this (it should ideally be just fn call_once(self) here).

[mark-i-m]

Question: Does this work?

const FOO: impl Foo = impl Foo { ... };

[burdges]

I believe the syntax by @eddyb makes more sense so long as you must still write out self arguments.

I prefer anonymous types being explicit though, so maybe some syntax involving ||? If I understand, you want closure style capture while creating a value, so maybe struct || impl Trait { ... } but I still do not see self working.

You could name the struct but infer the contents. In fact, if you only have type inference for struct fields, then

fn foo() -> impl Deref<Target=Bar>+DerefMut {
    ...
    #[derive(Clone)]
    struct Ret { x: _, y: _ };  // Or maybe struct Ret ||; 
    impl Deref for Ret {
        type Tagert = Bar;
        fn deref(&self) { self.x }
    }
    impl DerefMut for Ret {
        fn deref_mut(&mut self) { self.y }
    }
    Ret {x,y} // Or maybe Ret |x,y|;
}

In this, all captures take the form self.x, and all captured values must be explicitly passed, using field puns.

We could also improve the mechanisms for creating such types from procedural macros, so like some macro driven || Trait syntax

|| Iterator {
    type Item = Foo;
    fn next() -> Option<Foo> {  // self omitted since some macro processes this 
        ...
    }
}

There is also the FnBorrow traits suggestion which captures many use cases here, with the only strange syntax being some new named lifetime.

[jdahlstrom]

Yeah, this is basically exactly Java's anonymous inner classes. Somewhat amusing that for 20 years Java had closures but only using the verbose AIC syntax. Then they finally added a terse lambda syntax as a syntactic sugar (conceptually if not implementation-wise) . And now in Rust we first had lambdas and now entertain the thought of generalizing them :)

[mikeyhew]

It would be nice to see more examples where the trait is not one of the Fn* traits. The one showing how closures desugar to this syntax is useful as a sort of reference, but examples of new things you could do would be more motivating in my opinion.

[burdges]

Rust closures admit no polymorphism, not even lifetime polymorphism.

I suggested FnBorrow* traits largely because I wanted to call .iter_mut() multiple times, which requires shrinking a lifetime to the receiver's lifetime, ala FnMut() -> IterMut<'self,T>. In essence FnBorrow* would be this proposal, but far more ergonomic, and restricted to Borrow/BorrowMut, AsRef/AsMut, or Deref/DerrefMut`.

It's less common but you might want to tie a closure return lifetime to an argument lifetime too. At that point, you want full lifetime polymorphism for closures.

I think this proposal addresses roughly the same problems as polymorphism for closures. It gains some ergonomics in type descriptions via existing traits, but only with a dramatic ergonomics sacrifice for instantiation.

[Centril]

@burdges

Rust closures admit no polymorphism, not even lifetime polymorphism.

fn foo() {
    rank_2(|x: &u8| -> &u8 { x });
}

fn rank_2(x: impl for<'a> Fn(&'a u8) -> &'a u8) {}

[rodrimati1992]

for< > could be reused to declare closures like this:

use std::iter;

let cyclic=for<I:Iterator+Clone> |i:I|{
     i.cycle()
};


let list_1=cyclic(vec![1,2,3].into_iter())
    .take(9)
    .collect::<Vec<_>>();

assert_eq!(
    list_1,
    vec![1,2,3,1,2,3,1,2,3]
);


let list_2=cyclic(10..=12)
    .take(9)
    .collect::<Vec<_>>();

assert_eq!(
    list_2,
    vec![10,11,12,10,11,12,10,11,12]
);

[burdges]

Oops thanks :) It's only the receiver lifetime that's unreachable then.

[eddyb]

Oops thanks :) It's only the receiver lifetime that's unreachable then.

Yes and that's from the traits, not the closures. I think with type Output<'a>; and fn call(&'a self, ...) -> Self::Output<'a>;, that usecase would "just work".
(cc @nikomatsakis I'm not sure how close the borrow-checker is to supporting this)

[burdges]

How would Fn* traits look? I suppose the suggestion of a parallel FnBorrowMut and FnBorrow hierarchy is based on a miss-understanding, but maybe constraining Self in FnOnce::call_once works:

pub trait FnOnce<Args<'i>> {
    type Output<'o>;
    extern "rust-call" fn call_once<'s>(self, args: Args<'s>) -> Self::Output<'s> where Self: 's;
}
pub trait FnMut<Args<'i>>: FnOnce<Args<'i>> {
    extern "rust-call" fn call_mut<'s>(&'s mut self, args: Args<'s>) -> Self::Output<'s>;
}
pub trait Fn<Args<'i>>: FnMut<Args<'i>> {
    extern "rust-call" fn call<'s>(&'s self, args: Args<'s>) -> Self::Output<'s>;
}

impl<F> FnOnce<Args> for F where F: FnMut<Args> {
    extern "rust-call" fn call_once<'s>(self, args: Args<'s>) -> Self::Output<'s> where Self: 's {
        self.call_mut(args)
    }
}
impl<F> FnMut<Args> for F where F: Fn<Args> {
    extern "rust-call" fn call_mut<'s>(&'s mut self, args: Args<'s>) -> Self::Output<'s> {
        self.call(args)
    }
}

I'm still dubious this first impl makes sense. We'd need to inline it before borrow checking like a macro, right?

Also, there are no plans to support ATCs as type arguments like this, right? Could Args instead have an associated lifetime?

pub trait ArgsLifetime { type 'args; }

pub trait FnOnce<Args: ArgsLifetime> {
    type Output<'o>;
    extern "rust-call" fn call_once<'s>(self, args: Args) -> Self::Output<'s>
    where Self: 's, Args::'args: 's;
}
pub trait FnMut<Args>: FnOnce<Args> {
    extern "rust-call" fn call_mut<'s>(&'s mut self, args: Args) -> Self::Output<'s>
    where Args::'args: 's;
}
pub trait Fn<Args>: FnMut<Args> {
    extern "rust-call" fn call<'s>(&'s self, args: Args<'s>) -> Self::Output<'s>
    where Args::'args: 's;
}

impl<F> FnOnce<Args> for F where F: FnMut<Args> {
    extern "rust-call" fn call_once<'s>(self, args: Args) -> Self::Output<'s>
    where Self: 's, Args::'args: 's {
        self.call_mut(args)
    }
}
impl<F> FnMut<Args> for F where F: Fn<Args> {
    extern "rust-call" fn call_mut<'s>(&'s mut self, args: Args<'s>) -> Self::Output<'s>
    where Args::'args: 's {
        self.call(args)
    }
}

We'd still have the syntactic problem of naming this lifetime 's in Fn traits.

In the context of this RFC, if this worked then Fn* traits become far more powerful than AsRef/AsMut, IntoIterator, etc. because tricks like foo(|| x.lock()) or bar(|| y.iter_mut()) work if you write

fn foo<F>(f: F) where F: Fn() -> impl Deref<Target=..>+DerefMut { .. }
fn bar<F>(f: F) where F: Fn() -> impl Iterator<Target=..> { .. }

[eddyb]

You don't need to do anything for arguments, e.g. for<'a> Fn(&'a T) -> &'a U already exists.

type Output<'a>; only adds one more lifetime the output can depend on, on top of, lifetimes in the argument types and in Self.

[burdges]

I'd think that, if you add only Output<'a>, then Fn(&'a T) -> &'self U can be expressed, but not Fn(&'self T) -> &'self U. In particular, you cannot do

foo(xs.iter_mut().map(|x| [x,&mut y]));

because the 'a in [&'a T] must be either the closure call's 'self for &mut y or else 'x for the x: &'x mut T. We'd want the closure calls to shrink them to agree, so its Args needs to depend on 'self. Right?

[eddyb]

You can shorten any lifetime parameters of the impl (e.g. 'x) to what you call 'self.
So you'd just have type Output<'a> = [&'a T; 2];.

[Centril]

This line of discussion seems increasingly off-topic...

[bill-myers]

I think this should support implementing multiple traits (even if they have items with the same signature), and also support deriving traits.

[rodrimati1992]

How does this handle implementing a trait with supertraits?For example implementing std::iter::DoubleEndedIterator.

[H2CO3]

I see very little value in this. The example in the RFC is trivial to rewrite as

let world = "world";
let says_hello_world = {
    struct SaysHello<'a>(&'a str);
    impl<'a> fmt::Display for SaysHello<'a> {
        fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
            write!(f, "hello {}", self.0)
        }
    }
    SaysHello(world)
};
println!("{}", says_hello_world);

As others have already mentioned it, this is very similar to a lambda. Lambdas are great; however, the use-case of this one is so narrow and so specific that having to do the above transformation by hand once in a lifetime clearly doesn't justify the added language complexity.

[ExpHP]

Lambdas are great; however, the use-case of this one is so narrow and so specific that having to do the above transformation by hand once in a lifetime clearly doesn't justify the added language complexity.

1. This can be highly obnoxious. You need to redeclare all type parameters (named differently, because Rust forbids them to shadow!), duplicate all of the where bounds in scope, name the types of everything in its context, and at the end of all that you gotta construct it.

   • A trailing expression has to go at the end of a function, but the struct item declaration seems out of place unless it is above the impl. Thus: crap, gotta look way up there at the struct def so I can construct it properly

   • A lot of the fields in the constructor are bound to be barely inches away from being field-name punnable:

     Visitor {
         items: &items,
         name: &name,
     }

   • If you're upgrading an existing closure to a trait with more methods, now all of the upvars in your closure that were captured by reference need to be replaced with actual references. This changes the types in your code and can be a pain. match sugar and std library trait impls on & types help somewhat.

   • Naming the types of all the fields isn't always even possible.

2. APIs in Rust are currently forced to make a significant choice between ergonomics and power. If you accept F: FnMut, then you can only accept fns and closures, because the Fn traits can't be implemented manually. If you take F: MyTrait to allow nameable implementors (or e.g. to simulate for<T: Trait> Fn(&T) using a trait with a generic associated method), all user code now needs to write manual impls.

   • good luck writing a macro that can automate this to any comfortable degree for functions with context!

3. I simply don't see how the use case is "narrow and specific" (unless you refer specifically to the limitations of this proposal)? I can think of all sorts of places that commonly require this pattern:

   • AST traversal.

   • My code uses them to implement chemical potentials; returning a trait is like returning a bag of closures that can share data (e.g. for caching purposes/incremental computation. Computing these functions is expensive).

   • Any time a function wants to accept multiple callbacks that can be mutably closed over the same data (so multiple FnMuts won't do), e.g. for reporting various types of events that occur during an algorithm:

     struct LongThingVisitor<W>(W, BTreeMap<Id, ProgressBar>);
     
     impl<W: io::Write> LongThingVisitor for W {
         fn item_began(&mut self, item: &Item, id: Id) {
             unimplemented!("add to a list of fancy progress bars")
         }
     
         fn item_progress(&mut self, id: Id, progress: f64) {
             unimplemented!("update a progress bar")
         }
     
         fn item_finished(&mut self, id: Id) {
             unimplemented!("remove from the list")
         }
     }

[H2CO3]

A lot of the fields in the constructor are bound to be barely inches away from being field-name punnable:

That's a moot point, you can use a tuple struct.

APIs in Rust are currently forced to make a significant choice between ergonomics and power.

This is simply false. This would mean that Rust is simply a pain to write in, a viewpoint from which I beg to differ.

If you accept F: FnMut, then you can only accept fns and closures, because the Fn traits can't be implemented manually.

They are unstable. I would be highly in favor of stabilizing manual impl Fn for T in order to explicitly construct non-lambda callable types.

I simply don't see how the use case is "narrow and specific".

It's narrow and specific because it replicates a small subset of functionality already offered by other parts of the language, but only for one concrete situation/style. I understand that there are situations that it might be useful. But "there are situations where it's useful" isn't at all sufficient justification for any feature to be added to the language.

returning a trait is like returning a bag of closures that can share data

In your example, this doesn't really require such impl Trait expressions, only that impl Trait be allowed as the type of a binding if you don't want to allow the specific types to be relied on.

[burdges]

I suppose "narrow and specific" might be strongly phrased @ExpHP but the syntax mentioned by @H2CO3 is not much more complex than the syntax proposed here, and the syntax proposed here costs "strangeness budget".

We're talking about declaring arbitrary traits here so one should not expect dramatic simplifications. In particular you'll never navigate the orphan rules without those where clauses. In other words, I'd consider this RFC "narrow and specific" because many traits will remain painful to implement this way, so crate authors must trade their preferred trait structure for one that "lambdas well".

I'd think the most useful approach would be giving proc macros the information they require to capture like closures do, maybe eventually even doing a demo implementation of closures via such proc macros. There are likely a small number of traits, or trait combinations, that benefit enormously form this, so their authors could designed finely tuned constructions, rather than hack up their traits to make using this RFC's approach less painful. Also, I'd think almost everyone would love to pass more type information to proc macros, although some complexity exist of course.

[haslersn]

Why don't we just make closures castable to trait implementations if the trait has exactly one method?

use std::fmt;
fn main() {
    let world = "world";
    let says_hello_world = | f: &mut fmt::Formatter | write!(f, "hello {}", world);
    println!("{}", &says_hello_world as &fmt::Display);
}

[sfackler]

Why don't we just make lambdas castable to trait implementations if the trait has exactly one method?

That would be a nice extension, but both approaches are valuable to have. For example, Java adopted this behavior (lambda definitions of "functional interfaces") in version 8, and has additionally always had support for the style of anonymous interface implementation described in this RFC.

There are some issues with the lambda approach around specifying which trait should be implemented that this doesn't have.

[haslersn]

There are some issues with the lambda approach around specifying which trait should be implemented that this doesn't have.

That would be specified by the cast. I added an example to my previous comment.

It feels for me like:

• If the trait has more then one method, then it's too much to do it inline.
• If you need to use the self argument, then that's because you need attributes which is also too much to do inline.

Therefore, having closures castable to trait implementations (that have exactly one method) would be enaugh.

[graydon]

Opposed. This adds significant cognitive burden to readers and does not solve an important problem. The existing mechanism is a special case that should stay a special case, not be generalized.

[nikomatsakis]

@rfcbot fcp postpone

I personally am generally positive on this idea and I'd like to thank @canndrew for taking the time to write up and open it. Nonetheless, I am moving to postpone for the time being. Let me explain.

First, why am I in favor of this idea? Well, because I've often found a desire for this feature. One example would be in Rayon, where we often have to make "one off structs" to implement the ProducerCallback trait. However, I think the time is "not ripe" for this idea quite yet, and it's also probable that this is not the syntax we want.

Why is the time not ripe? While the details of roadmap is still in play, I think it seems pretty clear that we are trending towards a "refine the language, finish up things in play" sort of year, and this seems like a clear expansion with insufficiently strong motivation.

(Along those lines, there are some alternative, more limited approaches that we might also consider. For example, my Rayon use case above would be better served via generic closures, which seem like they have kind of zero extra "cognitive load". A more aggressive step might be to take the Java approach of permitting || foo to expand to any "single function" trait.)

[rfcbot]

Team member @nikomatsakis has proposed to postpone this. The next step is review by the rest of the tagged team members:

• @Centril
• @aturon
• @cramertj
• @eddyb
• @joshtriplett
• @nikomatsakis
• @pnkfelix
• @scottmcm
• @withoutboats

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to postpone, as per the review above, is now complete.

By the power vested in me by Rust, I hereby postpone this RFC.