Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Add unboxed, abstract return types #105

Closed
wants to merge 1 commit into from

Conversation

@aturon
Copy link
Member

aturon commented Jun 3, 2014

No description provided.

@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

FWIW, I feel somewhat uneasy about the impl Trait syntax proposed here; bikeshedding welcome!

I am currently leaning toward the more conservative design detailed under "Alternatives".

In today's Rust, you can write a function signature like
````rust
fn consume_iter_static<I: Iterator<u8>>(iter: I)
fn consume_iter_dyanmic(iter: Box<Iterator<u8>>)

This comment has been minimized.

Copy link
@huonw

huonw Jun 3, 2014

Member

s/an/na/

out, which can be very painful. Unboxed abstract types only require writing the
trait bound.
* _Documentation_. In today's Rust, reading the documentation for the `Iterator`

This comment has been minimized.

Copy link
@emberian

emberian Jun 3, 2014

Member

I looove this benefit.

This comment has been minimized.

Copy link
@Valloric

Valloric Jun 3, 2014

This is very nice, agreed.

This comment has been minimized.

Copy link
@ticki

ticki Aug 23, 2015

Contributor

+1


This code is roughly equivalent to
````rust
pub struct Result_produce_iter_static(

This comment has been minimized.

Copy link
@huonw

huonw Jun 3, 2014

Member

Totally minor point, but I think this example would be clearer as just

struct Result_produce_iter_static {
    inner:  iter::Skip<...>
}
impl Iterator<int> for Result_produce_iter_static {
     fn next(&mut self) -> Option<int> { self.inner.next() }
}

(In particular, there's no runtime difference for using a so-called "newtype" vs a normal struct, unlike Haskell; and I've noticed that LLVM seems to optimise a plain struct better than tuple structs anyway.)


Just as is currently done for trait objects, the typechecker must ensure that
lifetime parameters are not stripped when using an unboxed abstract type.
For example (adapted from @glaebhoeri):

This comment has been minimized.

Copy link
@huonw

huonw Jun 3, 2014

Member

@glaebhoerl (with an L)

@nrc
Copy link
Member

nrc commented Jun 3, 2014

Could you explain how this (e.g., fn foo() -> Tr in you proposal, where Tr is a trait) is different from, e.g., fn foo<X: T>()-> X in current Rust. As far as I can see, your proposal allows the callee to specify the concrete type for X rather than the caller - are there other differences? Is there a benefit over using a default type parameter for X (other than better encapsulation)?

Am I correct in assuming that the encapsulation here is only at the programmer level? That is, from the compiler's point of view, the caller does know the concrete type?

_implicit_ type argument.
Using unboxed abstract types in arguments makes (simple) static and dynamic
dispatch syntactically closer:

This comment has been minimized.

Copy link
@nrc

nrc Jun 3, 2014

Member

I fear that this is a downside - the difference at the moment is relatively easy to explain. With this shorthand, I fear the syntax for static and dynamic dispatch is too similar. In other words, we break the principle of 'things which are different should look different'.

This comment has been minimized.

Copy link
@chris-morgan

chris-morgan Jun 4, 2014

Member

At present, the static form is clumsy to read or to write, and so many people go in the direction of the less efficient dynamic dispatch. I view the increase in similarity as an improvement.

This comment has been minimized.

Copy link
@bstrie

bstrie Jun 4, 2014

Contributor

I would like to know where you've observed people preferring dynamic dispatch just because of the syntax. The static dispatch syntax is more familiar to C++ programmers, and I'd expect them to reach for it first.

This comment has been minimized.

Copy link
@sfackler

sfackler Jun 4, 2014

Member

I've seen people with functions returning Box<Iterator> in IRC decently often. The only alternative right now is to write a type signature that's probably too complex for anyone new to Rust to even figure out:

pub struct PhfMapEntries<'a, T> {
    priv iter: iter::FilterMap<'a,
                               &'a Option<(&'static str, T)>,
                               (&'static str, &'a T),
                               slice::Items<'a, Option<(&'static str, T)>>>,
}
fn extend_dynamic(&mut self, iterator: &Iterator<T>)
````
It may be especially important for passing unboxed closures as arguments.

This comment has been minimized.

Copy link
@nrc

nrc Jun 3, 2014

Member

Could you expand on this please? (and/or give an example)

This comment has been minimized.

Copy link
@aturon

aturon Jun 3, 2014

Author Member

@nick29581 Expanded with example.

@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

@nick29581

fn foo<X: T>() -> X says "for any type X implementing T, I'll produce an X".

fn foo() -> impl T says "there's some hidden type X implementing T; I'll produce an X"

You cannot use generic types/default type parameters to get at the second meaning, because the point is that the function's code produces a single, concrete return type of its choosing.

From the compiler's point of view, what the caller knows depends on the stage of the compiler:

  • during typechecking, the caller knows only the trait bound, not the concrete type
  • during codegen, the caller knows the concrete type and generates statically-dispatched calls, etc.

This and other details are, I believe, covered in the RFC; let me know if it's not clear.

could provide different concrete iterator types for the first and second
components of the tuple.
### Structs and other compound types

This comment has been minimized.

Copy link
@huonw

huonw Jun 3, 2014

Member

Is there a concrete use-case for this? It seems rather more complicated and adds an entirely implicit place of monomorphisation, that is, writing

fn use_foo(f: Foo) {}

is actually a generic function and will create multiple instantiations in the binary (am I interpreting this correctly?), but there is absolutely no indication of this from the signature. Is it crazy to restrict it to something like

struct Foo<T: Set<u8>> {
    s: T
}

fn use_foo(f: Foo<impl Set<u8>>)

(I guess this means not special-casing these types particularly.)

This comment has been minimized.

Copy link
@aturon

aturon Jun 3, 2014

Author Member

@huonw I agree; I am uneasy about putting these in structs, and I don't have a strong use-case for it.

The main reason for including it in the proposal was to treat impl Trait consistently as something you can write anywhere a type goes. But the more I think about it, the more I like the conservative alternative I outline in the end: restricting this RFC to function return types, and using the syntax "_ : Trait" instead.

This comment has been minimized.

Copy link
@aturon

aturon Jun 3, 2014

Author Member

@nikomatsakis might want to jump in here -- he first suggested allowing impl Trait in structs, but I'm not sure if he had a concrete use-case in mind.

This comment has been minimized.

Copy link
@nikomatsakis

nikomatsakis Jun 4, 2014

Contributor

Not really. I think I was just pushing the idea to see how far it could go. The return value variation is interesting, though I think there is value in permitting it in argument position. We have precedent for having fn signatures have rich shorthands and I think it's served us fairly well.

@SiegeLord
Copy link

SiegeLord commented Jun 3, 2014

While I really like the idea of using impl Foo in places other than the return type (as removes the syntactic weight of the 'prefered' method of dispatch), the implicit parametrization in the struct example just rubs me the wrong way; is there no way to indicate to the reader what is happening there? In the case of function arguments, this also seems to preclude being able to specify type hints for these implicit type parameters. Or, would this work?

fn foo1(b: impl Foo) {}
foo1<Bar>()
fn foo2<T>(a: T, b: impl Foo) {}
foo2<Bar, Baz>() // T is set to Bar, implicit one is set to Baz

Also, just for complete clarity, does the & go before or after the impl?


# Summary

Allow functions to return types to return _unboxed abstract types_, written

This comment has been minimized.

Copy link
@pczarn

pczarn Jun 3, 2014

remove to return types

@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

Thanks for the quick feedback; I've updated the RFC to respond to most of the points made. I'll also respond in comments.

@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

Added notes to the RFC on an additional choice: allowing impl Trait only in function signatures.

@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

@SiegeLord I very much agree with your concerns about allowing impl Trait to used everywhere, especially for the struct example. Honestly, I think allowing impl Trait in struct fields is probably a bad idea :-)

On the other hand, I've added an alternative design where impl Trait is only permitted in function signatures, which keeps the lightweight syntax but also means you can tell exactly where monomorphization is happening. Probably that design could allow explicitly supplying the concrete types for an impl Trait as well (as you're proposing).

Finally, regarding &, I think you'd want & impl Trait for "A reference to some T where T: Trait".

@huonw
Copy link
Member

huonw commented Jun 3, 2014

I don't see any mention of multiple traits, e.g.

fn foo() -> impl Iterator<int> + Clone

It's probably worth mentioning even if it's not explicitly part of this RFC.

fn collect_to_set<T, I: Iterator<T>>(iter: I) -> impl Set<T>
````
we could allow naming the concrete result type by a path like
`collect_to_set::<T, I>::impl`. The only way to get a value of this type is by

This comment has been minimized.

Copy link
@huonw

huonw Jun 3, 2014

Member

On first glance, I like this idea, especially since it makes the equality/self thing fall out automatically.

This comment has been minimized.

Copy link
@huonw

huonw Jun 4, 2014

Member

Although, there's a slight complication, what about something like

fn nested() -> Vec<impl Foo>

There is extra structure here, so presumably the nested::impl type would preferably point to the interior of the Vec rather than the whole return type (i.e. it's returning Vec<nested::impl>, meaning one might wish to write something like let x: &nested::impl = nested().get(0)), which then makes it hard to refer to values with multiple abstract generics, e.g.

fn tuple() -> (impl Iterator<int>, impl Iterator<u8>)

Also, what about abstract generics nested in others:

fn nested2() -> impl Iterator<impl Foo>
@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

@huonw Just added a brief section in "Unresolved questions" on multiple bounds. I think it should definitely be part of the design, but I'm not sure about the syntax. I seem to recall some problems recently regarding + when not inside < > braces?

Anyway, if it can work, my preferred syntax would be impl Trait1 + Trait2 as you proposed.

@huonw
Copy link
Member

huonw commented Jun 3, 2014

I seem to recall some problems recently regarding + when not inside < > braces?

I think this was with as, something like

foo as X + Y

is ambiguous as (foo as X) + Y or foo as (X+Y).

Which brings us onto another thing, would/should/could explicit some_value as impl Iterator<int> casts be useful?

@aturon
Copy link
Member Author

aturon commented Jun 3, 2014

@huonw OK, so + for multiple bounds syntax would likely be a problem only if we allow these casts.

I can't offhand see why you'd need such a cast form. I suppose that the RFC implicitly assumes that T can be used for impl Trait whenever T: Trait, without any explicit casts. (Essentially the same behavior you get when instantiating generics today.)

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Jun 4, 2014

I think the meaning of foo as impl Bar would be pretty much the same as using impl in a local variable: essentially an assertion that the type does implement Bar. Not sure if there is much point, though, it could never help your program compile in particular, just make it fail.

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Jun 4, 2014

(Also, the notation impl Foo + Bar is no more of a problem than the Foo + Bar types are today.)

@huonw
Copy link
Member

huonw commented Jun 4, 2014

(Also, the notation impl Foo + Bar is no more of a problem than the Foo + Bar types are today.)

Which Foo + Bar types?

@eddyb
Copy link
Member

eddyb commented Jun 4, 2014

Is there any specific reason the impl keyword is required?
I don't think we'll have a way to return unsized types (because of the technical challenges of doing so, given existing calling conventions), so why not use just the trait name?

That would allow us to write:

fn add(x: int) -> |int| -> int {|y| x + y}
@thestinger
Copy link

thestinger commented Jun 4, 2014

@eddyb: That's what I suggested in my unboxed closure proposal, and I don't see a problem with doing it like that.

@huonw
Copy link
Member

huonw commented Jun 4, 2014

Taking an anonymous generic as a parameter would be ambiguous with trait objects, e.g.

fn foo(x: &mut Trait) { ... }

could either be a trait object or equivalent to fn foo<T: Trait>(x: &mut T) { ... }.

@nagisa
Copy link
Contributor

nagisa commented Sep 8, 2015

fn abcd() ⇝ Trait

Here you go: a sigil, not intrusive, hard to spot and most importantly… extremely hard to type.


What I’m trying to say: this RFC was closed more than a year ago and nobody’s getting this feature in without a new RFC. As (my) previous experience shows, people tend to to go all over the same debate on a follow-up/replacement/new RFCs (see the placement-in + <-), therefore discussing syntax for a feature that’s not landing anytime soon looks like mostly a waste of time (people ignore discussions on previous RFCs most of the time).

@mitchmindtree
Copy link

mitchmindtree commented Sep 8, 2015

@nagisa I think most people are aware a new RFC is needed but are just getting some bikeshedding done before the time comes.

@eddyb
Copy link
Member

eddyb commented Sep 8, 2015

@nagisa I've waited more than a month for @aturon and then we had a meeting last week and a new RFC might be coming this month, if that gives you any hope.
The syntax problem is one that will create bikeshed before and after a new RFC, and I'd be glad if we had something optimal before then.

@ticki
Copy link
Contributor

ticki commented Sep 8, 2015

There is a point in making it short, simple, and easy, though, to encourage the programmer to use this over Box (when possible). Naming it AnAbstractUnboxedReturnType does not encourage this. A sigil, on the other side, can also create confusions.

@aturon
Copy link
Member Author

aturon commented Sep 8, 2015

@eddyb

I've waited more than a month for @aturon and then we had a meeting last week and a new RFC might be coming this month, if that gives you any hope.

Sorry! :(

I have a couple other RFCs in my queue, but hope to push them out this week, and then will focus on reviving this one. Thanks again, @eddyb, for your work on this topic.

@mitchmindtree
Copy link

mitchmindtree commented Sep 8, 2015

Here here! Thanks @eddyb and @aturon (and everyone else involved). Personally, this has been one of my most anticipated features - looking forward to seeing what unfolds 👍

@critiqjo
Copy link

critiqjo commented Sep 20, 2015

The syntax problem is one that will create bikeshed before and after a new RFC, and I'd be glad if we had something optimal before then.

So here's my two cents: I liked the impl Trait syntax, but I feel uneasy that it doesn't play well with the where clause, and things may get really long. So I propose an alternative, inspired from the pattern matching syntax:

fn factory(num: i32) -> T @ _
    where T : Fn(i32) -> i32
{
    move |x| x + num
}

I also liked what @Stebalien proposed here, and here's an alternative:

type X = T @ Arc<_> where T : Send;

during compilation X should be resolvable to a single concrete type.

The downside is that you have to use where clause now!

Update: another downside is that it is not obvious from the syntax that T is not a trait... So I guess using impl is the better choice...

@eddyb
Copy link
Member

eddyb commented Sep 20, 2015

@critiqjo But that has the wrong semantics: you're requiring that the type implement certain traits but not exposing it.
This is similar to the confusion with generics, and your syntax examples are less ergonomic than even the -> _: Trait syntax, which is the opposite direction of where I'd like this to go.

@critiqjo
Copy link

critiqjo commented Sep 20, 2015

That's true... 😞 But my original concern was to have an option to specify it using where...

@eddyb
Copy link
Member

eddyb commented Sep 20, 2015

@critiqjo I honestly don't see the point, what would where bring?
If it was actual existential syntax, it might be interesting, but it's pretty hard to use existentials for fn declarations because the whole fn is existential, not the return type (which would be a plain -> Trait).

@critiqjo
Copy link

critiqjo commented Sep 20, 2015

Wow! I see!! (I thought where was just to improve readability... Sorry for the noise...)

@aturon
Copy link
Member Author

aturon commented Sep 29, 2015

Thanks @glaebhoerl @Ericson2314 @eddyb and others for the insightful discussion since this RFC was closed. I've been thinking about this a fair amount, and after digesting your various comments, wrote up a blog post outlining a couple possible directions.

@Stebalien
Copy link
Contributor

Stebalien commented Sep 29, 2015

Nice! Syntax nit. I'd prefer the following over the arrow syntax:

trait IterAdapter: Iterator
    where Self: Clone if Self::Inner: Clone,
          Self: DoubleEndedIterator if Self::Inner: DoubleEndedIterator
{
    type Inner: Iterator;
}

To keep APIs sane, I wouldn't allow the inline version.

Also, this alone probably deserves its own RFC (it seems like it would be useful by itself).

@mitchmindtree
Copy link

mitchmindtree commented Sep 29, 2015

Just thought I'd mention there's further discussion of @aturon 's latest blogpost on reddit also.

@glaebhoerl
Copy link
Contributor

glaebhoerl commented Oct 3, 2015

@aturon (Going to respond here, because this is where most of the technical discussion has been, and the reddit discussion has fallen off the front pages by now.)

Here are some things which occurred to me while re-reading your post:

If I have my druthers, this feature would also be usable in argument position:

fn map<U>(self, f: ~FnOnce(T) -> U) -> Option<U>

However we end up solving the "abstract return type" use case, I agree it would be nice if it could extend to abstract arguments as well: it bothers me that we currently have to perform the same kind of fn hof<F: FnOnce(...)>(f: F) dance as C++. We should be able to do better.

That said, given the "leaky" semantics of the proposed ~Trait syntax, it seems to me that its analogous behavior in argument position would be much closer to C++ template expansion than to our existing generics:

fn my_print(thing: ~Display) {
    println!("{}", thing + 1)
}
my_print(21); // OK, prints 22
my_print("hi"); // compile error

Here, the fact that my_print can depend on the actual type that the caller chose to call it with, rather than just the specified Display interface, is simply dual to how given fn my_printable() -> ~Display, the caller could depend on the type that my_printable chose -- in either case meaning that a change in the implementation can break clients even while the signatures stay the same.

(Personally, this bothers me quite a bit: this is a question of priorities, but explicit interfaces and non-leaky abstractions would be much closer to hard requirements on my list, along with a clean, orthogonal design.)

  • It behaves exactly like associated types today.

[...]

  • This kind of “leakage” is already prevalent – and important! – in Rust today. For example, when you define an abstract type, you give a trait bound which must be fulfilled. But when a client has narrowed to a particular impl, everything about the associated type is revealed:

Could you spell this analogy out in greater detail? I don't quite have the intuition behind it. (One difference I notice is that with associated types, you do write out the actual type in at least one place, unlike with ~Trait - but it's not obvious to me what it corresponds to in the broader analogy.)

  • The type leakage is, in general, very unlikely to be relied upon. For example, to observe the particulars of an iterator adapter type, you’d have to do something like assign it to a suitably-typed mutable variable:

    let iter: Chain<Map<'a, (int, u8), u16, Enumerate<Filter<'a, u8, vec::MoveItems<u8>>>>, SkipWhile<'a, u16, Map<'a, &u16, u16, slice::Items<u16>>>>;
    iter = some_function();
    

I don't understand this example... why couldn't you rely on type inference? Why is mutability relevant? Either way, I don't think I agree with the broader point. On the one hand, maybe this is the case to some extent for iterator adapters, simply because these are special-purpose types whose only purpose in life is to adapt iterators, and there's inherently not much else you can do with them. But in general, most types have much broader interfaces. And on the other hand, I thought leakage for things like conditional impls was the whole point!

The basic idea is to introduce a “type abstraction operator” @ that is used to “seal” a concrete type to a particular interface:

pub type File = FileDesc@(Read + Write + Seek + Debug);

This is an intriguing approach, but you don't quite spell it out in the post -- what's the motivation for formulating things this way, rather than e.g. abstract type File: Read + Write + Seek + Debug = FileDesc?

  • How should these type definitions interact with coherence? Can you implement traits for File? Inherent methods? What if they conflict with traits/methods on FileDesc?

The answer feels like it should be "no", or at least, the rules should be akin to the ones for normal type aliases. You definitely shouldn't be able to give conflicting impls for File and FileDesc -- the owning module, at least, should see these as the same type. I guess it's an interesting question that if you do impl Foo for FileDesc and impl Bar for File, and if both File and FileDesc are exported, then outside the module you should be able to know that FileDesc: Foo and File: Bar, but not FileDesc: Bar or File: Foo (which you'd know inside the module, given you know File = FileDesc). That seems logical enough at least for this simple example, but it's kind of subtle and weird, so it might be a better idea to just forbid trait impls directly on abstract types. (Inherent impls seem more desirable... of course you'd like to provide an external API of things you can do with the abstract type, that's kind of the point. While from the owning module's perspective, it should still behave the same as if you were impling on a type alias.)

  • How do you deal with bounds where the type isn’t in Self position? For example, there is also an impl of Read and Write for &File that should be exported.

With the abstract type formulation, at least, it seems natural to use a where clause -- pub abstract type File = FileDesc where File: Read+Write+Seek+Debug, &File: Read+Write;, or somesuch.

@mitchmindtree
Copy link

mitchmindtree commented Oct 4, 2015

🔔 Just a notice to any thread followers, a possible alternative is being discussed in kimundi's latest RFC.

Edit: here's the reddit discussion.

@nrc
Copy link
Member

nrc commented Jan 12, 2016

Some thoughts on impl trait here: http://ncameron.org/blog/abstract-return-types-aka-%60impl-trait%60/

One thing I don't address there, but think will work is allowing impls to use a concrete type where the trait uses impl Trait. Then allowing callers which know they have exactly that impl to use the concrete return type

@comex
Copy link

comex commented Jan 13, 2016

Wait, you want OIBITs to leak from the function body? That seems like an odd abstraction violation. Why would Sync or Send be different from any other trait in that respect? Isn't part of the motivation to allow stabilizing APIs where the concrete type may change in the future?

@eddyb
Copy link
Member

eddyb commented Jan 13, 2016

@comex The reasoning is the same as with private fields: they are not exposed in the public API but they affect OIBITs.

If we don't reflect OIBITs through impl Trait, they would have to be explicitly specified, which apart from being an annotation nightmare, it would also require support for conditional bounds (e.g. (Send if T: Sync) or (Sync if 'a: 'static)).
And it wouldn't compose, at all. If a new OIBIT is added, existing impl Trait uses wouldn't have it.

My only concerns were about it requiring global inference to implement, but I believe we can create "global obligations" that are checked after all the types are known.

The basic idea is to allow code like the following:
````rust
pub fn produce_iter_static() -> impl Iterator<int> {

This comment has been minimized.

Copy link
@gdox

gdox Sep 11, 2016

Maybe not the best place to write this, but the impl syntax sounds a bit confusing for me given the impl Trait for Struct syntax we currently have. What about (something like) the following?

pub fn produce_iter_static<I>() -> I guarantees I : Iterator<int> {
    range(0, 10).rev().map(|x| x * 2).skip(2)
}

This way, the usual syntax of static dispatch (the where-clause) is kept.
As a bonus, it allows:

pub fn produce_iter_static<I>() -> I guarantees I : Iterator<int> + Clone {...}

This comment has been minimized.

Copy link
@Mark-Simulacrum

Mark-Simulacrum Sep 11, 2016

Member

@gdox This should be discussed in the tracking issue.

wycats pushed a commit to wycats/rust-rfcs that referenced this pull request Mar 5, 2019
ember-cli should look up for addons in optionalDependencies as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.