RFC: Add unboxed, abstract return types

[aturon]

No description provided.

[aturon]

FWIW, I feel somewhat uneasy about the impl Trait syntax proposed here; bikeshedding welcome!

I am currently leaning toward the more conservative design detailed under "Alternatives".

[huonw]

s/an/na/

[cmr]

I looove this benefit.

[Valloric]

This is very nice, agreed.

[ticki]

+1

[huonw]

Totally minor point, but I think this example would be clearer as just

struct Result_produce_iter_static {
    inner:  iter::Skip<...>
}
impl Iterator<int> for Result_produce_iter_static {
     fn next(&mut self) -> Option<int> { self.inner.next() }
}

(In particular, there's no runtime difference for using a so-called "newtype" vs a normal struct, unlike Haskell; and I've noticed that LLVM seems to optimise a plain struct better than tuple structs anyway.)

[huonw]

@glaebhoerl (with an L)

[nrc]

Could you explain how this (e.g., fn foo() -> Tr in you proposal, where Tr is a trait) is different from, e.g., fn foo<X: T>()-> X in current Rust. As far as I can see, your proposal allows the callee to specify the concrete type for X rather than the caller - are there other differences? Is there a benefit over using a default type parameter for X (other than better encapsulation)?

Am I correct in assuming that the encapsulation here is only at the programmer level? That is, from the compiler's point of view, the caller does know the concrete type?

[nrc]

I fear that this is a downside - the difference at the moment is relatively easy to explain. With this shorthand, I fear the syntax for static and dynamic dispatch is too similar. In other words, we break the principle of 'things which are different should look different'.

[chris-morgan]

At present, the static form is clumsy to read or to write, and so many people go in the direction of the less efficient dynamic dispatch. I view the increase in similarity as an improvement.

[bstrie]

I would like to know where you've observed people preferring dynamic dispatch just because of the syntax. The static dispatch syntax is more familiar to C++ programmers, and I'd expect them to reach for it first.

[sfackler]

I've seen people with functions returning Box<Iterator> in IRC decently often. The only alternative right now is to write a type signature that's probably too complex for anyone new to Rust to even figure out:

pub struct PhfMapEntries<'a, T> {
    priv iter: iter::FilterMap<'a,
                               &'a Option<(&'static str, T)>,
                               (&'static str, &'a T),
                               slice::Items<'a, Option<(&'static str, T)>>>,
}

[nrc]

Could you expand on this please? (and/or give an example)

[aturon]

@nick29581 Expanded with example.

[aturon]

@nick29581

fn foo<X: T>() -> X says "for any type X implementing T, I'll produce an X".

fn foo() -> impl T says "there's some hidden type X implementing T; I'll produce an X"

You cannot use generic types/default type parameters to get at the second meaning, because the point is that the function's code produces a single, concrete return type of its choosing.

From the compiler's point of view, what the caller knows depends on the stage of the compiler:

• during typechecking, the caller knows only the trait bound, not the concrete type
• during codegen, the caller knows the concrete type and generates statically-dispatched calls, etc.

This and other details are, I believe, covered in the RFC; let me know if it's not clear.

[huonw]

Is there a concrete use-case for this? It seems rather more complicated and adds an entirely implicit place of monomorphisation, that is, writing

fn use_foo(f: Foo) {}

is actually a generic function and will create multiple instantiations in the binary (am I interpreting this correctly?), but there is absolutely no indication of this from the signature. Is it crazy to restrict it to something like

struct Foo<T: Set<u8>> {
    s: T
}

fn use_foo(f: Foo<impl Set<u8>>)

(I guess this means not special-casing these types particularly.)

[aturon]

@huonw I agree; I am uneasy about putting these in structs, and I don't have a strong use-case for it.

The main reason for including it in the proposal was to treat impl Trait consistently as something you can write anywhere a type goes. But the more I think about it, the more I like the conservative alternative I outline in the end: restricting this RFC to function return types, and using the syntax "_ : Trait" instead.

[aturon]

@nikomatsakis might want to jump in here -- he first suggested allowing impl Trait in structs, but I'm not sure if he had a concrete use-case in mind.

[nikomatsakis]

Not really. I think I was just pushing the idea to see how far it could go. The return value variation is interesting, though I think there is value in permitting it in argument position. We have precedent for having fn signatures have rich shorthands and I think it's served us fairly well.

[SiegeLord]

While I really like the idea of using impl Foo in places other than the return type (as removes the syntactic weight of the 'prefered' method of dispatch), the implicit parametrization in the struct example just rubs me the wrong way; is there no way to indicate to the reader what is happening there? In the case of function arguments, this also seems to preclude being able to specify type hints for these implicit type parameters. Or, would this work?

fn foo1(b: impl Foo) {}
foo1<Bar>()
fn foo2<T>(a: T, b: impl Foo) {}
foo2<Bar, Baz>() // T is set to Bar, implicit one is set to Baz

Also, just for complete clarity, does the & go before or after the impl?

[pczarn]

remove to return types

[aturon]

Thanks for the quick feedback; I've updated the RFC to respond to most of the points made. I'll also respond in comments.

[aturon]

Added notes to the RFC on an additional choice: allowing impl Trait only in function signatures.

[aturon]

@SiegeLord I very much agree with your concerns about allowing impl Trait to used everywhere, especially for the struct example. Honestly, I think allowing impl Trait in struct fields is probably a bad idea :-)

On the other hand, I've added an alternative design where impl Trait is only permitted in function signatures, which keeps the lightweight syntax but also means you can tell exactly where monomorphization is happening. Probably that design could allow explicitly supplying the concrete types for an impl Trait as well (as you're proposing).

Finally, regarding &, I think you'd want & impl Trait for "A reference to some T where T: Trait".

[huonw]

I don't see any mention of multiple traits, e.g.

fn foo() -> impl Iterator<int> + Clone

It's probably worth mentioning even if it's not explicitly part of this RFC.

[huonw]

On first glance, I like this idea, especially since it makes the equality/self thing fall out automatically.

[huonw]

Although, there's a slight complication, what about something like

fn nested() -> Vec<impl Foo>

There is extra structure here, so presumably the nested::impl type would preferably point to the interior of the Vec rather than the whole return type (i.e. it's returning Vec<nested::impl>, meaning one might wish to write something like let x: &nested::impl = nested().get(0)), which then makes it hard to refer to values with multiple abstract generics, e.g.

fn tuple() -> (impl Iterator<int>, impl Iterator<u8>)

Also, what about abstract generics nested in others:

fn nested2() -> impl Iterator<impl Foo>

[aturon]

@huonw Just added a brief section in "Unresolved questions" on multiple bounds. I think it should definitely be part of the design, but I'm not sure about the syntax. I seem to recall some problems recently regarding + when not inside < > braces?

Anyway, if it can work, my preferred syntax would be impl Trait1 + Trait2 as you proposed.

[huonw]

I seem to recall some problems recently regarding + when not inside < > braces?

I think this was with as, something like

foo as X + Y

is ambiguous as (foo as X) + Y or foo as (X+Y).

Which brings us onto another thing, would/should/could explicit some_value as impl Iterator<int> casts be useful?

[aturon]

@huonw OK, so + for multiple bounds syntax would likely be a problem only if we allow these casts.

I can't offhand see why you'd need such a cast form. I suppose that the RFC implicitly assumes that T can be used for impl Trait whenever T: Trait, without any explicit casts. (Essentially the same behavior you get when instantiating generics today.)

[nikomatsakis]

I think the meaning of foo as impl Bar would be pretty much the same as using impl in a local variable: essentially an assertion that the type does implement Bar. Not sure if there is much point, though, it could never help your program compile in particular, just make it fail.

[nikomatsakis]

(Also, the notation impl Foo + Bar is no more of a problem than the Foo + Bar types are today.)

[huonw]

(Also, the notation impl Foo + Bar is no more of a problem than the Foo + Bar types are today.)

Which Foo + Bar types?

[eddyb]

Is there any specific reason the impl keyword is required?
I don't think we'll have a way to return unsized types (because of the technical challenges of doing so, given existing calling conventions), so why not use just the trait name?

That would allow us to write:

fn add(x: int) -> |int| -> int {|y| x + y}

[thestinger]

@eddyb: That's what I suggested in my unboxed closure proposal, and I don't see a problem with doing it like that.

[huonw]

Taking an anonymous generic as a parameter would be ambiguous with trait objects, e.g.

fn foo(x: &mut Trait) { ... }

could either be a trait object or equivalent to fn foo<T: Trait>(x: &mut T) { ... }.

[nagisa]

fn abcd() ⇝ Trait

Here you go: a sigil, not intrusive, hard to spot and most importantly… extremely hard to type.

---

What I’m trying to say: this RFC was closed more than a year ago and nobody’s getting this feature in without a new RFC. As (my) previous experience shows, people tend to to go all over the same debate on a follow-up/replacement/new RFCs (see the placement-in + <-), therefore discussing syntax for a feature that’s not landing anytime soon looks like mostly a waste of time (people ignore discussions on previous RFCs most of the time).

[mitchmindtree]

@nagisa I think most people are aware a new RFC is needed but are just getting some bikeshedding done before the time comes.

[eddyb]

@nagisa I've waited more than a month for @aturon and then we had a meeting last week and a new RFC might be coming this month, if that gives you any hope.
The syntax problem is one that will create bikeshed before and after a new RFC, and I'd be glad if we had something optimal before then.

[ticki]

There is a point in making it short, simple, and easy, though, to encourage the programmer to use this over Box (when possible). Naming it AnAbstractUnboxedReturnType does not encourage this. A sigil, on the other side, can also create confusions.

[aturon]

@eddyb

I've waited more than a month for @aturon and then we had a meeting last week and a new RFC might be coming this month, if that gives you any hope.

Sorry! :(

I have a couple other RFCs in my queue, but hope to push them out this week, and then will focus on reviving this one. Thanks again, @eddyb, for your work on this topic.

[mitchmindtree]

Here here! Thanks @eddyb and @aturon (and everyone else involved). Personally, this has been one of my most anticipated features - looking forward to seeing what unfolds 👍

[critiqjo]

The syntax problem is one that will create bikeshed before and after a new RFC, and I'd be glad if we had something optimal before then.

So here's my two cents: I liked the impl Trait syntax, but I feel uneasy that it doesn't play well with the where clause, and things may get really long. So I propose an alternative, inspired from the pattern matching syntax:

fn factory(num: i32) -> T @ _
    where T : Fn(i32) -> i32
{
    move |x| x + num
}

I also liked what @Stebalien proposed here, and here's an alternative:

type X = T @ Arc<_> where T : Send;

during compilation X should be resolvable to a single concrete type.

The downside is that you have to use where clause now!

Update: another downside is that it is not obvious from the syntax that T is not a trait... So I guess using impl is the better choice...

[eddyb]

@critiqjo But that has the wrong semantics: you're requiring that the type implement certain traits but not exposing it.
This is similar to the confusion with generics, and your syntax examples are less ergonomic than even the -> _: Trait syntax, which is the opposite direction of where I'd like this to go.

[critiqjo]

That's true... 😞 But my original concern was to have an option to specify it using where...

[eddyb]

@critiqjo I honestly don't see the point, what would where bring?
If it was actual existential syntax, it might be interesting, but it's pretty hard to use existentials for fn declarations because the whole fn is existential, not the return type (which would be a plain -> Trait).

[critiqjo]

Wow! I see!! (I thought where was just to improve readability... Sorry for the noise...)

[aturon]

Thanks @glaebhoerl @Ericson2314 @eddyb and others for the insightful discussion since this RFC was closed. I've been thinking about this a fair amount, and after digesting your various comments, wrote up a blog post outlining a couple possible directions.

[Stebalien]

Nice! Syntax nit. I'd prefer the following over the arrow syntax:

trait IterAdapter: Iterator
    where Self: Clone if Self::Inner: Clone,
          Self: DoubleEndedIterator if Self::Inner: DoubleEndedIterator
{
    type Inner: Iterator;
}

To keep APIs sane, I wouldn't allow the inline version.

Also, this alone probably deserves its own RFC (it seems like it would be useful by itself).

[mitchmindtree]

Just thought I'd mention there's further discussion of @aturon 's latest blogpost on reddit also.

[glaebhoerl]

@aturon (Going to respond here, because this is where most of the technical discussion has been, and the reddit discussion has fallen off the front pages by now.)

Here are some things which occurred to me while re-reading your post:

If I have my druthers, this feature would also be usable in argument position:

fn map<U>(self, f: ~FnOnce(T) -> U) -> Option<U>

However we end up solving the "abstract return type" use case, I agree it would be nice if it could extend to abstract arguments as well: it bothers me that we currently have to perform the same kind of fn hof<F: FnOnce(...)>(f: F) dance as C++. We should be able to do better.

That said, given the "leaky" semantics of the proposed ~Trait syntax, it seems to me that its analogous behavior in argument position would be much closer to C++ template expansion than to our existing generics:

fn my_print(thing: ~Display) {
    println!("{}", thing + 1)
}
my_print(21); // OK, prints 22
my_print("hi"); // compile error

Here, the fact that my_print can depend on the actual type that the caller chose to call it with, rather than just the specified Display interface, is simply dual to how given fn my_printable() -> ~Display, the caller could depend on the type that my_printable chose -- in either case meaning that a change in the implementation can break clients even while the signatures stay the same.

(Personally, this bothers me quite a bit: this is a question of priorities, but explicit interfaces and non-leaky abstractions would be much closer to hard requirements on my list, along with a clean, orthogonal design.)

• It behaves exactly like associated types today.

[...]

• This kind of “leakage” is already prevalent – and important! – in Rust today. For example, when you define an abstract type, you give a trait bound which must be fulfilled. But when a client has narrowed to a particular impl, everything about the associated type is revealed:

Could you spell this analogy out in greater detail? I don't quite have the intuition behind it. (One difference I notice is that with associated types, you do write out the actual type in at least one place, unlike with ~Trait - but it's not obvious to me what it corresponds to in the broader analogy.)

• The type leakage is, in general, very unlikely to be relied upon. For example, to observe the particulars of an iterator adapter type, you’d have to do something like assign it to a suitably-typed mutable variable:

  let iter: Chain<Map<'a, (int, u8), u16, Enumerate<Filter<'a, u8, vec::MoveItems<u8>>>>, SkipWhile<'a, u16, Map<'a, &u16, u16, slice::Items<u16>>>>;
  iter = some_function();
  

I don't understand this example... why couldn't you rely on type inference? Why is mutability relevant? Either way, I don't think I agree with the broader point. On the one hand, maybe this is the case to some extent for iterator adapters, simply because these are special-purpose types whose only purpose in life is to adapt iterators, and there's inherently not much else you can do with them. But in general, most types have much broader interfaces. And on the other hand, I thought leakage for things like conditional impls was the whole point!

The basic idea is to introduce a “type abstraction operator” @ that is used to “seal” a concrete type to a particular interface:

pub type File = FileDesc@(Read + Write + Seek + Debug);

This is an intriguing approach, but you don't quite spell it out in the post -- what's the motivation for formulating things this way, rather than e.g. abstract type File: Read + Write + Seek + Debug = FileDesc?

• How should these type definitions interact with coherence? Can you implement traits for File? Inherent methods? What if they conflict with traits/methods on FileDesc?

The answer feels like it should be "no", or at least, the rules should be akin to the ones for normal type aliases. You definitely shouldn't be able to give conflicting impls for File and FileDesc -- the owning module, at least, should see these as the same type. I guess it's an interesting question that if you do impl Foo for FileDesc and impl Bar for File, and if both File and FileDesc are exported, then outside the module you should be able to know that FileDesc: Foo and File: Bar, but not FileDesc: Bar or File: Foo (which you'd know inside the module, given you know File = FileDesc). That seems logical enough at least for this simple example, but it's kind of subtle and weird, so it might be a better idea to just forbid trait impls directly on abstract types. (Inherent impls seem more desirable... of course you'd like to provide an external API of things you can do with the abstract type, that's kind of the point. While from the owning module's perspective, it should still behave the same as if you were impling on a type alias.)

• How do you deal with bounds where the type isn’t in Self position? For example, there is also an impl of Read and Write for &File that should be exported.

With the abstract type formulation, at least, it seems natural to use a where clause -- pub abstract type File = FileDesc where File: Read+Write+Seek+Debug, &File: Read+Write;, or somesuch.

[mitchmindtree]

🔔 Just a notice to any thread followers, a possible alternative is being discussed in kimundi's latest RFC.

Edit: here's the reddit discussion.

[nrc]

Some thoughts on impl trait here: http://ncameron.org/blog/abstract-return-types-aka-%60impl-trait%60/

One thing I don't address there, but think will work is allowing impls to use a concrete type where the trait uses impl Trait. Then allowing callers which know they have exactly that impl to use the concrete return type

[comex]

Wait, you want OIBITs to leak from the function body? That seems like an odd abstraction violation. Why would Sync or Send be different from any other trait in that respect? Isn't part of the motivation to allow stabilizing APIs where the concrete type may change in the future?

[eddyb]

@comex The reasoning is the same as with private fields: they are not exposed in the public API but they affect OIBITs.

If we don't reflect OIBITs through impl Trait, they would have to be explicitly specified, which apart from being an annotation nightmare, it would also require support for conditional bounds (e.g. (Send if T: Sync) or (Sync if 'a: 'static)).
And it wouldn't compose, at all. If a new OIBIT is added, existing impl Trait uses wouldn't have it.

My only concerns were about it requiring global inference to implement, but I believe we can create "global obligations" that are checked after all the types are known.

[gdox]

Maybe not the best place to write this, but the impl syntax sounds a bit confusing for me given the impl Trait for Struct syntax we currently have. What about (something like) the following?

pub fn produce_iter_static<I>() -> I guarantees I : Iterator<int> {
    range(0, 10).rev().map(|x| x * 2).skip(2)
}

This way, the usual syntax of static dispatch (the where-clause) is kept.
As a bonus, it allows:

pub fn produce_iter_static<I>() -> I guarantees I : Iterator<int> + Clone {...}

[Mark-Simulacrum]

@gdox This should be discussed in the tracking issue.