Generic associated types (associated type constructors)

[withoutboats]

I definitely realize that this feature may not be prioritized right now, and I expect that some aspects of it probably present real implementation challenges (especially partial application of type operators and extending HRTBs), but I wanted to document and discuss what seems to me like the best incremental step toward higher-kinded polymorphism in Rust.

Rendered

[edited to update rendered link —mbrubeck]

[kennytm]

There is a missing ` after the "not".

[P1start]

This syntax feels a bit too magic to me—&[u8] almost always acts as &'a [u8] with 'a elided, and it’s not immediately obvious at a glance that Item is actually a higher-order type of kind & -> *. Also, what if multiple lifetime parameters are omitted? Is &&T equivalent to <'a, 'b> => &'a &'b T or <'a, 'b> => &'b &'a T?

I would prefer a more obvious (but more verbose) syntax: StreamingIterator<Item<'a>=&'a [u8]>. That way it’s apparent that 'a is declared as part of the trait bound, and it also nicely matches the syntax for associated item definitions.

[glaebhoerl]

That also works better if the associated type is generic over a type rather than a lifetime -- Foo<Bar<T>=&'static [T]> is certainly more obvious than Foo<Bar=&'static []>, for example.

[arielb1]

It certainly fits the pattern of lifetime elision done today - fn(&[u8]) is today for<'a> fn(&'a [u8]), so Stream<Item=&[u8]> is Stream<Item=λ'a. &'a [u8]>. Of course we probably want a fully-explicit way of writing these things.

By the way, I would not like to use & to refer to lifetime - it has a perfectly good meaning of reference today.

[eddyb]

By the way, I would not like to use & to refer to lifetime - it has a perfectly good meaning of reference today.

I agree, why not '*?

[krdln]

The "elision" syntax (I mean StreamingIterator<Item=&[u8]>) makes sense for '* → * (& → *) types and I don't think it would create confusion or ambiguity. But for bigger number of input params or non-lifetime inputs I can't image how it would work, so we need a non-elided syntax too. I thought of reusing the closure syntax (Item = |'a| &'a[u8]), but the one proposed by @P1start (Item<'a>=&'a [u8]) looks more Rusty imho.

@withoutboats You are talking about allowing any operators, but the syntax seems only defined for constructors. If you plan to support any operators, could you give an example of using * → * → * associated operator (not (*, *) → *)? Also, an example for type as an input would be nice too. I know it's analogous to lifetime as an input, but it would be nice to have it for completeness. An example would be also nice for multi-input constructor.

[eddyb]

@krdln The "type-level closure syntax" already exists, and the example would be Item = for<'a> &'a [u8], although I prefer Item<'a> = &'a [u8] myself.

[krdln]

@eddyb The (implements) for<'a> 'a[u8] means ∀'a (implements) &'a[u8]. The (implements) for<'a> Trait<'a> means ∀'a (implements) Trait<'a>. I think it's kind-of analogous, but a totally different thing than a type-lambda. And it would be confusing to use the same syntax for those two things.

If you squint, you can imagine that for<'a> Trait<'a> has a type signature of (& → * → bool) → (* → bool). Which means "take a lifetime parametrised (& → ...) trait (* → bool) and return a trait (* → bool)" * → bool (a trait bound). And that's a totally different thing than & → *.

[eddyb]

T: for<'a> Trait<'a> is sugar for for<'a> T: Trait<'a>, i.e. the universal quantification covers the whole bound.
I suppose it would be a bit weird if the following two snippets were equivalent:

type Foo = for<'a> fn(&'a T) -> &'a U;
type Bar = for<'a> Trait<'a>;

type Foo<'a> = fn(&'a T) -> &'a U;
type Bar<'a> = Trait<'a>;

And if omitting explicit <'x> would mean universal quantification, which conflicts with anonymous lifetimes in a function signature and/or elision.
Although, this is only a problem with lifetimes AFAICT, unless there are situations where you can actually have a Box<for<T: Foo> Trait<T>>.

[withoutboats]

I would prefer a more obvious (but more verbose) syntax: StreamingIterator<Item<'a>=&'a [u8]>. That way it’s apparent that 'a is declared as part of the trait bound, and it also nicely matches the syntax for associated item definitions.

I agree that this syntax is quite magic (and is easily the least satisfying syntax here to me), but I'm concerned about how much this looks like some sort of higher-rank syntax. If we do add an ability to make these parameters higher-rank, I am a bit concerned about the difference in meaning about 'a between Item<'a>=&'a [u8] and for<'a> Item<'a>: Display.

But for bigger number of input params or non-lifetime inputs I can't image how it would work, so we need a non-elided syntax too.

You're right, the elision syntax is right out for that reason.

By the way, I would not like to use & to refer to lifetime - it has a perfectly good meaning of reference today.

This syntax is not internal to Rust of course. I find '* hard to distinguish from *. I think the best syntax might be the clearest, using words: lifetime -> type.

[glaebhoerl]

I think the best syntax might be the clearest, using words: lifetime -> type.

I agree!

Not even Haskellers really like their * syntax for "type" and in GHC 8 they're getting Type as an alias for it.

[glaebhoerl]

I agree that this seems like the ideal starting point for adding higher-kinded capability to Rust. Even the idea of moving forward incrementally in this way, as opposed to "Adding Higher-Kinded Types" all at once in a "big bang", is something that hadn't occurred to me, but seems obvious in hindsight.

To stick with our own terminology consistently, instead having of a mishmash of vocabulary from Rust, Haskell, and the academic literature, I would probably prefer to call this feature "generic associated types" or "associated generic types". (I would also prefer to call "higher-kinded types" instead "higher-order generic types", contrasting with "higher-rank types" which are "higher-order generic functions", but the usage of "HKT" is incredibly widespread by this point.)

I also suspect that most of the difficult design work which needs to be done before this feature can move forward is on how to extend the type system to support it, which this RFC (self-admittedly) doesn't even attempt to address. Am I right?

[glaebhoerl]

I agree that this seems like the ideal starting point for adding higher-kinded capability to Rust. Even the idea of moving forward incrementally in this way, as opposed to "Adding Higher-Kinded Types" all at once in a "big bang", is something that hadn't occurred to me, but seems obvious in hindsight.

To stick with our own terminology consistently, instead having of a mishmash of vocabulary from Rust, Haskell, and the academic literature, I would probably prefer to call this feature "generic associated types" or "associated generic types". (I would also prefer to call "higher-kinded types" instead "higher-order generic types", contrasting with "higher-rank types" which are "higher-order generic functions", but the usage of "HKT" is incredibly widespread by this point.)

I also suspect that most of the difficult design work which needs to be done before this feature can move forward is on how to extend the type system to support it, which this RFC (self-admittedly) doesn't even attempt to address. Am I right?

[dwrensha]

Currently, an approximation of StreamingIterator can be defined like this:

trait StreamingIterator<'a> {
  type Item: 'a;
  fn next(&'a self) -> Option<Self::Item>;
}

You can then bound types as T: for<'a> StreamingIterator<'a> to avoid the lifetime parameter infecting everything StreamingIterator appears.

However, this only partially prevents the infectiveness of StreamingIterator, only allows for some of the types that associated type operators can express, and is in generally a hacky attempt to work around the limitation rather than an equivalent alternative.

I'm interested in seeing a concrete example of something that cannot be expressed using this hack. I've hit situations where I've wanted associated type operators as proposed in this RFC, but in every case it has sufficed hoist the the type parameters up to the trait itself. See, for example, these capnp traits. I agree that such hoisting can be ugly, and I feel like there are probably cases where it fails, but those cases are not immediately obvious to me.

[dwrensha]

Currently, an approximation of StreamingIterator can be defined like this:

trait StreamingIterator<'a> {
  type Item: 'a;
  fn next(&'a self) -> Option<Self::Item>;
}

You can then bound types as T: for<'a> StreamingIterator<'a> to avoid the lifetime parameter infecting everything StreamingIterator appears.

However, this only partially prevents the infectiveness of StreamingIterator, only allows for some of the types that associated type operators can express, and is in generally a hacky attempt to work around the limitation rather than an equivalent alternative.

I'm interested in seeing a concrete example of something that cannot be expressed using this hack. I've hit situations where I've wanted associated type operators as proposed in this RFC, but in every case it has sufficed hoist the the type parameters up to the trait itself. See, for example, these capnp traits. I agree that such hoisting can be ugly, and I feel like there are probably cases where it fails, but those cases are not immediately obvious to me.

[BurntSushi]

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

Ideally, the StreamingIterator trait should be expanded to contain many of the same utility adapter methods that the Iterator trait has.

[BurntSushi]

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

Ideally, the StreamingIterator trait should be expanded to contain many of the same utility adapter methods that the Iterator trait has.

[eddyb]

AFAIK the bulk of the implementation work needed here is removing the constraints on the "3 application levels" in the current compiler (type/trait parameters, Self and fn/method parameters).

The actual resolution of non-monomorphic associated types won't actually be that big of a problem, we already have to do that sort of transformation for generic methods.

And lastly, what I'm worried most about is checking uses of such a trait with polymorphic associated types without providing concrete bindings.
We've had a few stages of adjustment to all the problems associated types can cause in various edge cases.
I suspect this is the "difficult design work" @glaebhoerl was talking about, in which case I agree.

But overall, I like this RFC and @rust-lang/compiler will continue to work towards making such advances possible, regardless of which exact syntactic or semantic choices we end up accepting.

[eddyb]

AFAIK the bulk of the implementation work needed here is removing the constraints on the "3 application levels" in the current compiler (type/trait parameters, Self and fn/method parameters).

The actual resolution of non-monomorphic associated types won't actually be that big of a problem, we already have to do that sort of transformation for generic methods.

And lastly, what I'm worried most about is checking uses of such a trait with polymorphic associated types without providing concrete bindings.
We've had a few stages of adjustment to all the problems associated types can cause in various edge cases.
I suspect this is the "difficult design work" @glaebhoerl was talking about, in which case I agree.

But overall, I like this RFC and @rust-lang/compiler will continue to work towards making such advances possible, regardless of which exact syntactic or semantic choices we end up accepting.

[krdln]

@dwrensha
I think that it's currently impossible to express something like owning_ref, but generic over reference type. Eg. the interface of generic reference could look like:

unsafe trait Ref {
    type Item<'a>;
    unsafe fn erase_lifetime<'a>(x: Item<'a>) -> Item<'static> { transmute(x) }
    unsafe fn introduce_lifetime<'a>(x: Item<'static>) -> Item<'a> { transmute(x) }
}

The main benefit from this RFC here would be that compiler would know that all types created by the same & → * constructor are transmute-compatible, and that it can apply that constructor for any lifetime. Unfortunately, the trait would have to be implemented for some phantom type, since this RFC doesn't allow traits to be implemented for type constructors, but I think it's only a minor issue.

It seems that the main benefit from this RFC comes from allowing lifetime (& → *) constructors, so I'd be happy to see it accepted even if it doesn't allow full-featured * → * ones. (analogicaly, as we have for<'a> syntax only for lifetimes)

[krdln]

@dwrensha
I think that it's currently impossible to express something like owning_ref, but generic over reference type. Eg. the interface of generic reference could look like:

unsafe trait Ref {
    type Item<'a>;
    unsafe fn erase_lifetime<'a>(x: Item<'a>) -> Item<'static> { transmute(x) }
    unsafe fn introduce_lifetime<'a>(x: Item<'static>) -> Item<'a> { transmute(x) }
}

The main benefit from this RFC here would be that compiler would know that all types created by the same & → * constructor are transmute-compatible, and that it can apply that constructor for any lifetime. Unfortunately, the trait would have to be implemented for some phantom type, since this RFC doesn't allow traits to be implemented for type constructors, but I think it's only a minor issue.

It seems that the main benefit from this RFC comes from allowing lifetime (& → *) constructors, so I'd be happy to see it accepted even if it doesn't allow full-featured * → * ones. (analogicaly, as we have for<'a> syntax only for lifetimes)

[eddyb]

@krdln Actually, right now you can't even use transmute there, although I would be inclined to accept a same-type exception (we already erase lifetimes, so it should just see Item<'static> on both sides).

[eddyb]

@krdln Actually, right now you can't even use transmute there, although I would be inclined to accept a same-type exception (we already erase lifetimes, so it should just see Item<'static> on both sides).

[krdln]

@eddyb Your example doesn't compile, since you can't apply an lifetime paremeter to Self::Item. Maybe you've meant to use Ref<'a> somewhere there? I've tried, but I couldn't make it compile. So I'm still not convinced that thing is possible in today's Rust (but I won't be surprised if it was possible, Rust can be really surprising).

Using your suggestion, I came up with this, so this use case might be supported in current Rust, with some fixes for transmute behaviour.

[krdln]

@eddyb Your example doesn't compile, since you can't apply an lifetime paremeter to Self::Item. Maybe you've meant to use Ref<'a> somewhere there? I've tried, but I couldn't make it compile. So I'm still not convinced that thing is possible in today's Rust (but I won't be surprised if it was possible, Rust can be really surprising).

Using your suggestion, I came up with this, so this use case might be supported in current Rust, with some fixes for transmute behaviour.

[eddyb]

@krdln Hah, that was dumb of me. Totally missed the fact that you were applying multiple times within the trait itself. It could work via free functions with R: Ref<'a> + Ref<'static> but that's probably not useful to you.

[eddyb]

@krdln Hah, that was dumb of me. Totally missed the fact that you were applying multiple times within the trait itself. It could work via free functions with R: Ref<'a> + Ref<'static> but that's probably not useful to you.

[soltanmm]

@krdln I think there does exist an injection between what is proposed here and the syntax that exists today, but it requires a few more pieces than what @dwrensha pointed out.

use std::mem;

// '* -> *
trait R2T<'a> {
    type T;
}

// * -> *
trait T2T<A> {
    type T;
}

// ('* -> *)('*)
type AppR<'a, T> where T: R2T<'a> = <T as R2T<'a>>::T;
// (* -> *)(*)
type AppT<A, T> where T: T2T<A> = <T as T2T<A>>::T;

// A trait with a higher kinded operator
unsafe trait Ref {
    type Item: for<'a> R2T<'a>;
    unsafe fn erase_lifetime<'a>(x: AppR<'a, Self::Item>) -> AppR<'static, Self::Item> { mem::transmute(x) }
    unsafe fn introduce_lifetime<'a>(x: AppR<'static, Self::Item>) -> AppR<'a, Self::Item> { mem::transmute(x) }
}

// A type with a '* -> '* -> * HKT
trait MyType
    where
        for<'a> Self::BiRefItem: R2T<'a>,
        for<'a, 'b> AppR<'a, Self::BiRefItem>: R2T<'b>
{
    type BiRefItem;
}

// Use of partial application by currying (which can always be extended to full partial application with helpers)
trait PartiallyApplied<'a, T: MyType>
    where
        Self::Ref: Ref<Item=AppR<'a, T::BiRefItem>>
{
    type Ref;
    // ...
}

fn main() {}

However, it is excessively verbose, highly error prone, works only for lifetimes in certain use-cases (because it depends on HRTBs), and there're some issues with the compiler that prevent it from compiling (see rust-lang/rust#30472 and rust-lang/rust#31580).

For a more practical application of the general approach that does work today, see #1403 (comment).

For background on the general approach, see the fine-print in the associated items RFC. n.b. the RFC uses non-existent syntax in its example.

^{and I am in no way suggesting that one take a syntax-sugar approach implementing this :-P}

[soltanmm]

@krdln I think there does exist an injection between what is proposed here and the syntax that exists today, but it requires a few more pieces than what @dwrensha pointed out.

use std::mem;

// '* -> *
trait R2T<'a> {
    type T;
}

// * -> *
trait T2T<A> {
    type T;
}

// ('* -> *)('*)
type AppR<'a, T> where T: R2T<'a> = <T as R2T<'a>>::T;
// (* -> *)(*)
type AppT<A, T> where T: T2T<A> = <T as T2T<A>>::T;

// A trait with a higher kinded operator
unsafe trait Ref {
    type Item: for<'a> R2T<'a>;
    unsafe fn erase_lifetime<'a>(x: AppR<'a, Self::Item>) -> AppR<'static, Self::Item> { mem::transmute(x) }
    unsafe fn introduce_lifetime<'a>(x: AppR<'static, Self::Item>) -> AppR<'a, Self::Item> { mem::transmute(x) }
}

// A type with a '* -> '* -> * HKT
trait MyType
    where
        for<'a> Self::BiRefItem: R2T<'a>,
        for<'a, 'b> AppR<'a, Self::BiRefItem>: R2T<'b>
{
    type BiRefItem;
}

// Use of partial application by currying (which can always be extended to full partial application with helpers)
trait PartiallyApplied<'a, T: MyType>
    where
        Self::Ref: Ref<Item=AppR<'a, T::BiRefItem>>
{
    type Ref;
    // ...
}

fn main() {}

However, it is excessively verbose, highly error prone, works only for lifetimes in certain use-cases (because it depends on HRTBs), and there're some issues with the compiler that prevent it from compiling (see rust-lang/rust#30472 and rust-lang/rust#31580).

For a more practical application of the general approach that does work today, see #1403 (comment).

For background on the general approach, see the fine-print in the associated items RFC. n.b. the RFC uses non-existent syntax in its example.

^{and I am in no way suggesting that one take a syntax-sugar approach implementing this :-P}

[soltanmm]

Actually, now that I think about it... Unless this RFC is limited to kinds that look like '* -> *, doesn't it require extending HRTBs to types (in mechanism, not necessarily in syntax) to be useful? Otherwise we can't put bounds on the outputs of HKTs like * -> *. Seems like doing that first could be a reasonable alternative.

[soltanmm]

Actually, now that I think about it... Unless this RFC is limited to kinds that look like '* -> *, doesn't it require extending HRTBs to types (in mechanism, not necessarily in syntax) to be useful? Otherwise we can't put bounds on the outputs of HKTs like * -> *. Seems like doing that first could be a reasonable alternative.

[eddyb]

@soltanmm It is limited to '* -> *. However, this brings an interesting point I didn't realize: T: Trait<Item<'a> = &'a [u8]> is sugar for T: Trait and for<'a> <T as Trait>::Item<'a> = &'a [u8] so still HRTB in nature, just packaged away neatly.

[eddyb]

@soltanmm It is limited to '* -> *. However, this brings an interesting point I didn't realize: T: Trait<Item<'a> = &'a [u8]> is sugar for T: Trait and for<'a> <T as Trait>::Item<'a> = &'a [u8] so still HRTB in nature, just packaged away neatly.

[withoutboats]

To stick with our own terminology consistently, instead having of a mishmash of vocabulary from Rust, Haskell, and the academic literature, I would probably prefer to call this feature "generic associated types" or "associated generic types". (I would also prefer to call "higher-kinded types" instead "higher-order generic types", contrasting with "higher-rank types" which are "higher-order generic functions", but the usage of "HKT" is incredibly widespread by this point.)

I don't feel strongly about the terminology, but I'll note that I avoided the term "higher-kinded type" in this RFC because I found it very confusing before I read TaPL, which avoids that terminology. A big part of my confusion was that calling type operators "types" suggests to me that this is a language feature that allows one to create values of a "type" of a higher kinded, like let x: Vec.

I also don't think we use the term generics that much, do we? I usually see 'type parameter' or 'parameterized type.'

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

I think others have answered this well (the most obvious thing is: * -> *), but I want to add that the real point is to make this feature easier and more intuitive to use, just as adding const parameters would be an easier easier and more intuitive feature than the type-level peano number hacks that we have right now.

You are talking about allowing any operators, but the syntax seems only defined for constructors. If you plan to support any operators, could you give an example of using * → * → * associated operator (not (*, *) → *)?

I used the term operator throughout the text, but its correct that this would only allow associated type constructors.

Actually, now that I think about it... Unless this RFC is limited to kinds that look like '* -> *, doesn't it require extending HRTBs to types (in mechanism, not necessarily in syntax) to be useful? Otherwise we can't put bounds on the outputs of HKTs like * -> *. Seems like doing that first could be a reasonable alternative.

I don't know anything about how HRTBs are implemented, some of eddyb's comments suggest this might be an easy extension, but the higher-ranked parameter is being applied in a different position in these two expressions:

for<'a> Type<'a>: Trait
Type: for<'a> Trait<'a>

This RFC explicitly says it doesn't propose allowing for<'a> Type<'a>: Trait, though obviously that would be the most useful thing to add next. I this RFC is useful even if you can't bound this new kind of associated item.

Also, this RFC only discusses lifetime -> type associated items, but I don't think there's anything in the RFC that doesn't apply equivalently to all kinds of type constructor. The main reason lifetime -> type is the only kind discussed is its the only kind for which I know of obvious, valuable use cases. Not coincidentally, I think its also the kind which can be easily explained without type theory concepts: it lets the trait define "view" types for its receiver.

Extending HRTB to be used with associated items of a kind other than lifetime -> type would require more work, though, of course.

[withoutboats]

To stick with our own terminology consistently, instead having of a mishmash of vocabulary from Rust, Haskell, and the academic literature, I would probably prefer to call this feature "generic associated types" or "associated generic types". (I would also prefer to call "higher-kinded types" instead "higher-order generic types", contrasting with "higher-rank types" which are "higher-order generic functions", but the usage of "HKT" is incredibly widespread by this point.)

I don't feel strongly about the terminology, but I'll note that I avoided the term "higher-kinded type" in this RFC because I found it very confusing before I read TaPL, which avoids that terminology. A big part of my confusion was that calling type operators "types" suggests to me that this is a language feature that allows one to create values of a "type" of a higher kinded, like let x: Vec.

I also don't think we use the term generics that much, do we? I usually see 'type parameter' or 'parameterized type.'

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

I think others have answered this well (the most obvious thing is: * -> *), but I want to add that the real point is to make this feature easier and more intuitive to use, just as adding const parameters would be an easier easier and more intuitive feature than the type-level peano number hacks that we have right now.

You are talking about allowing any operators, but the syntax seems only defined for constructors. If you plan to support any operators, could you give an example of using * → * → * associated operator (not (*, *) → *)?

I used the term operator throughout the text, but its correct that this would only allow associated type constructors.

Actually, now that I think about it... Unless this RFC is limited to kinds that look like '* -> *, doesn't it require extending HRTBs to types (in mechanism, not necessarily in syntax) to be useful? Otherwise we can't put bounds on the outputs of HKTs like * -> *. Seems like doing that first could be a reasonable alternative.

I don't know anything about how HRTBs are implemented, some of eddyb's comments suggest this might be an easy extension, but the higher-ranked parameter is being applied in a different position in these two expressions:

for<'a> Type<'a>: Trait
Type: for<'a> Trait<'a>

This RFC explicitly says it doesn't propose allowing for<'a> Type<'a>: Trait, though obviously that would be the most useful thing to add next. I this RFC is useful even if you can't bound this new kind of associated item.

Also, this RFC only discusses lifetime -> type associated items, but I don't think there's anything in the RFC that doesn't apply equivalently to all kinds of type constructor. The main reason lifetime -> type is the only kind discussed is its the only kind for which I know of obvious, valuable use cases. Not coincidentally, I think its also the kind which can be easily explained without type theory concepts: it lets the trait define "view" types for its receiver.

Extending HRTB to be used with associated items of a kind other than lifetime -> type would require more work, though, of course.

[withoutboats]

Thinking about the syntactic and semantic complexities of bounds using associated type constructors, and the fact that all of this seems to be conceivable as just an extension of HRTBs, I think it might be a good idea to not even discuss bounds using associated type constructors in this RFC, and instead delegate that to an RFC about extending HRTBs.

[withoutboats]

Thinking about the syntactic and semantic complexities of bounds using associated type constructors, and the fact that all of this seems to be conceivable as just an extension of HRTBs, I think it might be a good idea to not even discuss bounds using associated type constructors in this RFC, and instead delegate that to an RFC about extending HRTBs.

[eddyb]

@withoutboats Both of those forms are allowed right now and the for<'a> actually gets lifted to the whole bound (if you don't mean (for<'a> Type<'a>): Trait, of course).

In a way, for<'a> is only syntactical, at the semantic level debruijn indices are used for "late-bound lifetimes" with "binders" which define the boundaries where they apply.

So for<'a> (for<'b> fn(&'b T) -> (&'b U, &'a V)): Trait<'a> ends up being something like fn(&'2 T) -> (&'2 U, &'1 V): Trait<'1> with a binder around the signature of the fn type and another one around the whole bound.

it might be a good idea to not even discuss bounds using associated type constructors in this RFC

It's not a good idea to punt on this matter, because the T: Trait<Assoc = U> syntax is sugar for a <T as Trait>::Assoc = U bound (which we can't express right now syntactically, sadly).

[eddyb]

@withoutboats Both of those forms are allowed right now and the for<'a> actually gets lifted to the whole bound (if you don't mean (for<'a> Type<'a>): Trait, of course).

In a way, for<'a> is only syntactical, at the semantic level debruijn indices are used for "late-bound lifetimes" with "binders" which define the boundaries where they apply.

So for<'a> (for<'b> fn(&'b T) -> (&'b U, &'a V)): Trait<'a> ends up being something like fn(&'2 T) -> (&'2 U, &'1 V): Trait<'1> with a binder around the signature of the fn type and another one around the whole bound.

it might be a good idea to not even discuss bounds using associated type constructors in this RFC

It's not a good idea to punt on this matter, because the T: Trait<Assoc = U> syntax is sugar for a <T as Trait>::Assoc = U bound (which we can't express right now syntactically, sadly).

[withoutboats]

It's not a good idea to punt on this matter, because the T: Trait<Assoc = U> syntax is sugar for a <T as Trait>::Assoc = U bound (which we can't express right now syntactically, sadly).

I'm not sure why this makes it bad to punt, but I think the important takeaway is that we (yet) don't need to be able to put type constructors in where clauses, because we can use HRTB. Obviously this won't be sufficient for later use cases involving higher-kinded traits.

[withoutboats]

It's not a good idea to punt on this matter, because the T: Trait<Assoc = U> syntax is sugar for a <T as Trait>::Assoc = U bound (which we can't express right now syntactically, sadly).

I'm not sure why this makes it bad to punt, but I think the important takeaway is that we (yet) don't need to be able to put type constructors in where clauses, because we can use HRTB. Obviously this won't be sufficient for later use cases involving higher-kinded traits.

[withoutboats]

Another issue has occurred to me just now. We would probably also want some sort of implicit HRTBs at the declaration site of the associated type, as in:

trait Collection {
    type Elem;
    type Iter<'a>: Iterator<Item=&'a Self::Elem>;
    type IterMut<'a>: Iterator<Item=&'a mut Self::Elem>;
    ...
}

[withoutboats]

Another issue has occurred to me just now. We would probably also want some sort of implicit HRTBs at the declaration site of the associated type, as in:

trait Collection {
    type Elem;
    type Iter<'a>: Iterator<Item=&'a Self::Elem>;
    type IterMut<'a>: Iterator<Item=&'a mut Self::Elem>;
    ...
}

[eddyb]

@withoutboats I wonder if we should agree on using "polymorphic" or some other "painfully clear" term because I thought you meant <T as Trait>::PolyAssoc<'a>: RegularTrait instead of <T as Trait>::PolyAssoc: SomethingNew, and the latter is HKT territory as far as I'm concerned, so we're in violent agreement, I suppose.

[eddyb]

@withoutboats I wonder if we should agree on using "polymorphic" or some other "painfully clear" term because I thought you meant <T as Trait>::PolyAssoc<'a>: RegularTrait instead of <T as Trait>::PolyAssoc: SomethingNew, and the latter is HKT territory as far as I'm concerned, so we're in violent agreement, I suppose.

[withoutboats]

@eddyb: Right, the RFC currently discusses <T as Trait>::PolyAssoc=TypeConstructor, but instead of allowing that that we can just use for<'a> <T as Trait>::PolyAssoc<'a>=TypeConstructor<'a>

[withoutboats]

@eddyb: Right, the RFC currently discusses <T as Trait>::PolyAssoc=TypeConstructor, but instead of allowing that that we can just use for<'a> <T as Trait>::PolyAssoc<'a>=TypeConstructor<'a>

[withoutboats]

I've edited and expanded the RFC in response to the comments so far, mainly clarifications and adding content about how this would require HRTB changes.

[withoutboats]

I've edited and expanded the RFC in response to the comments so far, mainly clarifications and adding content about how this would require HRTB changes.

[soltanmm]

When's the RFC to spec out the syntax of bounds/where-clauses on HRTB-quantified types coming out? 😃

^{aaand obligatory cc #1481}

[soltanmm]

_{okay github hopefully you won't just eat my comment this time...}
I thought we determined otherwise. Or do you mean something outside the form of for<'x> T: U? If something different it's probably worth clarifying.

[withoutboats]

You're right, I misunderstood and thought that it was disallowed syntactically.

[eefriedman]

The syntax struct Foo<T> where T: for<'a> StreamingIterator<Item=&'a mut [u8]> from the example in the RFC doesn't seem right. The following expanded example is legal on nightly Rust:

trait StreamingIterator {
    type Item;
}
struct Foo<T> where T: for<'a> StreamingIterator<Item=&'a mut [u8]> {
    x: T
}

Granted, as far as I can tell there is no way to write T which satisfies this constraint at the moment, but as far as I can tell that's orthogonal to this RFC.

I think what you actually want is something closer to struct Foo<T> where T: StreamingIterator<Item<'a>=&'a mut [u8]>, with no for<>.

[eefriedman]

The syntax struct Foo<T> where T: for<'a> StreamingIterator<Item=&'a mut [u8]> from the example in the RFC doesn't seem right. The following expanded example is legal on nightly Rust:

trait StreamingIterator {
    type Item;
}
struct Foo<T> where T: for<'a> StreamingIterator<Item=&'a mut [u8]> {
    x: T
}

Granted, as far as I can tell there is no way to write T which satisfies this constraint at the moment, but as far as I can tell that's orthogonal to this RFC.

I think what you actually want is something closer to struct Foo<T> where T: StreamingIterator<Item<'a>=&'a mut [u8]>, with no for<>.

[withoutboats]

You're right that its wrong, but it needs both the <'a> and the for<'a>. Without the for<'a>, it would search the surrounding scope for a lifetime parameter called 'a and fail.

EDIT: Fixed.

[withoutboats]

You're right that its wrong, but it needs both the <'a> and the for<'a>. Without the for<'a>, it would search the surrounding scope for a lifetime parameter called 'a and fail.

EDIT: Fixed.

[eefriedman]

The syntax for declaring a constraint on a type constructor should be parallel to the syntax for declaring/implementing it. type Item<'a> = &'a mut [T]; introduces a lifetime name without for<>; it should be the same for bounds on associated types.

Another way of looking at it: for<> has the wrong scope. It declares a parameter to the trait as a whole, not the type constructor. Actually, you could end up in a situation where you want to write something along the lines of for<'a> Foo<&'a i32, Bar<'b>=(&'a i32, &'b i32)>.

[eefriedman]

The syntax for declaring a constraint on a type constructor should be parallel to the syntax for declaring/implementing it. type Item<'a> = &'a mut [T]; introduces a lifetime name without for<>; it should be the same for bounds on associated types.

Another way of looking at it: for<> has the wrong scope. It declares a parameter to the trait as a whole, not the type constructor. Actually, you could end up in a situation where you want to write something along the lines of for<'a> Foo<&'a i32, Bar<'b>=(&'a i32, &'b i32)>.

[soltanmm]

@eefriedman If I understand your suggestion correctly that disallows constructs like:

trait Foo {
    type Bar<'x> where for<'x> Self::Bar<'x>: Trait1, Self::Bar<'static>: Trait1 + Trait2;
}

Is that intentional?

[soltanmm]

@eefriedman If I understand your suggestion correctly that disallows constructs like:

trait Foo {
    type Bar<'x> where for<'x> Self::Bar<'x>: Trait1, Self::Bar<'static>: Trait1 + Trait2;
}

Is that intentional?

[eefriedman]

I don't think my suggestion interacts with where clauses in the way you're suggesting. As far as I can tell, the scope of 'x in type Bar<'x> where... doesn't extend beyond the where keyword, so that part of the RFC isn't affected.

Actually, I'm not sure I see the point of the syntax in general; your example is precisely equivalent to:

trait Foo where for<'x> Self::Bar<'x>: Trait1, Self::Bar<'static>: Trait1 + Trait2 {
    type Bar<'x>;
}

[eefriedman]

I don't think my suggestion interacts with where clauses in the way you're suggesting. As far as I can tell, the scope of 'x in type Bar<'x> where... doesn't extend beyond the where keyword, so that part of the RFC isn't affected.

Actually, I'm not sure I see the point of the syntax in general; your example is precisely equivalent to:

trait Foo where for<'x> Self::Bar<'x>: Trait1, Self::Bar<'static>: Trait1 + Trait2 {
    type Bar<'x>;
}

[eefriedman]

Oh, sorry, I think I see what you mean. struct Foo<T> where T: for<'a> StreamingIterator<Item<'a>=&'a mut [u8]> is supposed to parallel something like struct Foo<T> where T: StreamingIterator, for<'a> <T as StreamingIterator>::Item<'a> = &'a mut [u8], where the for<> is clearly necessary. I was thinking of it as equality of the type constructor rather than equality of the resulting type. I guess either way works.

[eefriedman]

Oh, sorry, I think I see what you mean. struct Foo<T> where T: for<'a> StreamingIterator<Item<'a>=&'a mut [u8]> is supposed to parallel something like struct Foo<T> where T: StreamingIterator, for<'a> <T as StreamingIterator>::Item<'a> = &'a mut [u8], where the for<> is clearly necessary. I was thinking of it as equality of the type constructor rather than equality of the resulting type. I guess either way works.

[withoutboats]

@eefriedman your comments are a little unclear to me, perhaps I have misunderstood. However, the bound and the declaration should not be parallel, because they don't mean the same thing.

type Item<'a> = &'a mut [T]

This declares a binding to a type constructor, which has the kind lifetime -> type, and assigns it to a particular type constructor which has that kind. It is analogous to a function declaration, and the 'a here is analogous to the argument to a function.

for<'a> Self::Item<'a> = &'a mut [T]`.

This is saying that "for all lifetimes 'a, Self::Item applied to 'a is equivalent to &mut [T] applied to 'a." Its analogous to comparing the result of two function applications, and the 'a here is a parameter that is introduced to be applied

That is, this is like something like this, but at the type level:

// Declaration and assignment (type Item<'a> = &'a mut [T])
let square = |x| x * x;

// Introduce a variable (for<'a>)
for x in every_valid_x {
   // Bound assertion (Self::Item<'a> = &'a mut [T])
   assert_eq!(square(x), x * x);
}

This RFC doesn't propose allowing you to bound by type constructors themselves, only by the result of applying those constructors to arguments introduces somewhere (either in a higher-rank for or in the outer scope).

EDIT: It looks like the confusion got cleared up while I was writing this, but I'll leave this comment in case it helps other folks understand the difference.

[withoutboats]

@eefriedman your comments are a little unclear to me, perhaps I have misunderstood. However, the bound and the declaration should not be parallel, because they don't mean the same thing.

type Item<'a> = &'a mut [T]

This declares a binding to a type constructor, which has the kind lifetime -> type, and assigns it to a particular type constructor which has that kind. It is analogous to a function declaration, and the 'a here is analogous to the argument to a function.

for<'a> Self::Item<'a> = &'a mut [T]`.

This is saying that "for all lifetimes 'a, Self::Item applied to 'a is equivalent to &mut [T] applied to 'a." Its analogous to comparing the result of two function applications, and the 'a here is a parameter that is introduced to be applied

That is, this is like something like this, but at the type level:

// Declaration and assignment (type Item<'a> = &'a mut [T])
let square = |x| x * x;

// Introduce a variable (for<'a>)
for x in every_valid_x {
   // Bound assertion (Self::Item<'a> = &'a mut [T])
   assert_eq!(square(x), x * x);
}

This RFC doesn't propose allowing you to bound by type constructors themselves, only by the result of applying those constructors to arguments introduces somewhere (either in a higher-rank for or in the outer scope).

EDIT: It looks like the confusion got cleared up while I was writing this, but I'll leave this comment in case it helps other folks understand the difference.

[withoutboats]

I was thinking about the Read and Write traits, and an alternative defintiion of these traits that delegated the question of whether or not they could fail to their implementers. Here's a design pattern with advantages and drawbacks:

trait Read {
    type Result<T> = io::Result<T>;
    fn read(&mut self, &mut [u8]) -> Self::Result<usize>;
    fn read_to_end(&mut self, &mut Vec<u8>) -> Self::Result<()>;
}

impl Read for for File {
    fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> { ... }
    fn read_to_end(&mut self, buf: &mut Vec<u8>) -> io::Result<()> { ... }
}

impl Read for [u8] {
    type Result<T> = T;
    fn read(&mut self, buf: &mut [u8]) -> usize { ... }
    fn read_to_end(&mut self, buf: &mut Vec<u8>) { ... }
}

There are some disadvantages to this and it may not be the right call for Read/Write (of course, its too late to even consider it for those traits anyway), but I think there are probably traits where this would be the right thing to do.

[withoutboats]

I was thinking about the Read and Write traits, and an alternative defintiion of these traits that delegated the question of whether or not they could fail to their implementers. Here's a design pattern with advantages and drawbacks:

trait Read {
    type Result<T> = io::Result<T>;
    fn read(&mut self, &mut [u8]) -> Self::Result<usize>;
    fn read_to_end(&mut self, &mut Vec<u8>) -> Self::Result<()>;
}

impl Read for for File {
    fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> { ... }
    fn read_to_end(&mut self, buf: &mut Vec<u8>) -> io::Result<()> { ... }
}

impl Read for [u8] {
    type Result<T> = T;
    fn read(&mut self, buf: &mut [u8]) -> usize { ... }
    fn read_to_end(&mut self, buf: &mut Vec<u8>) { ... }
}

There are some disadvantages to this and it may not be the right call for Read/Write (of course, its too late to even consider it for those traits anyway), but I think there are probably traits where this would be the right thing to do.

[withoutboats]

Despite using StreamingIterator as the motivating example, this RFC does not even attempt to solve the problem of how a streaming iterator could be used as an iterator.

[withoutboats]

This raises some questions which are not answered in this section of the RFC. On functions, where clauses make that function optional when the where clause is not met. That is not the case here. Is it a good idea to use internal where clauses with majorly different semantics?

Are these where clauses anything but sugar over adding to the where clause at the head of the trait? Are they restricted to HRTBs for this type constructor, or could they contain arbitrary additional clauses?

Is there a more fitting syntax for this feature which meets all of these conditions?

• The clause is clearly and easily connected to the associated type constructor (e.g. by being a part of the associated type constructor's declaration)
• The clause is unambiguously distinguished from higher-kinded trait bounds on the associated type constructor itself (for forward compatibility).
• The clause is self-evidently restricted to providing bounds on the types produced by this associated type constructor.

[eddyb]

@withoutboats I think something like ! / Void as the error type might be a better fit than HKT.

[eddyb]

@withoutboats I think something like ! / Void as the error type might be a better fit than HKT.

[withoutboats]

@eddyb I'm not sure what you mean, do you mean something like this:

trait Read {
    type Error = io::Error;
    fn read(&mut self, &mut [u8]) -> Result<usize, Self::Error>;
}

impl Read for [u8] {
    type Error = !;
    fn read(&mut self, &mut [u8]) -> Result<usize, !>;
}

The advantage of using HKTs is that you can avoid writing unwraps / having unused must_use warnings. The disadvantage of course is that the return type becomes undefined in generic contexts, or else you can't use the non-erroring types.

[withoutboats]

@eddyb I'm not sure what you mean, do you mean something like this:

trait Read {
    type Error = io::Error;
    fn read(&mut self, &mut [u8]) -> Result<usize, Self::Error>;
}

impl Read for [u8] {
    type Error = !;
    fn read(&mut self, &mut [u8]) -> Result<usize, !>;
}

The advantage of using HKTs is that you can avoid writing unwraps / having unused must_use warnings. The disadvantage of course is that the return type becomes undefined in generic contexts, or else you can't use the non-erroring types.

[rkjnsn]

With some gymnastics, am I correct that this would allow being generic over Rc/Arc? Specifically create a trait to cover common Rc/Arc operations, create a second trait with an associated type constructor bound by that trait, and implement that second trait for two dummy types, one corresponding to Rc and the other to Arc? Obviously, it would be easier if we just had (type -> type) -> type).

[rkjnsn]

With some gymnastics, am I correct that this would allow being generic over Rc/Arc? Specifically create a trait to cover common Rc/Arc operations, create a second trait with an associated type constructor bound by that trait, and implement that second trait for two dummy types, one corresponding to Rc and the other to Arc? Obviously, it would be easier if we just had (type -> type) -> type).

[withoutboats]

I'm afraid not. Because this RFC doesn't include higher-kinded traits, you can't bound an associated type constructor by anything but the type that constructor evaluates to when applied to a type variable. Specifically, the trait covering common Rc/Arc operations wouldn't be able to express anything you couldn't express right now.

[withoutboats]

I'm afraid not. Because this RFC doesn't include higher-kinded traits, you can't bound an associated type constructor by anything but the type that constructor evaluates to when applied to a type variable. Specifically, the trait covering common Rc/Arc operations wouldn't be able to express anything you couldn't express right now.

[withoutboats]

@theduke It means the other team members haven't reviewed it yet.

[withoutboats]

@theduke It means the other team members haven't reviewed it yet.

[dobkeratops]

just reading through this, very interesting; according to your current thinking, (i) what would the closest approximation to haskell Functor / fmap look like, (ii) would this be able to handle abstraction over smart-pointer pointer-types too (I'm a bit hazy on where the boundaries are .. I'm not a haskell expert by any means; i'm familiar with using 'template-template parameters' in C++ and dabled with haskell a bit more since using rust. If I've understood right you seem to say this isa stepping stone to 'full hkt' that handles some popular cases ). I guess you prefer to talk about iterators rather than a clone of 'fmap' because they implement laziness/composability in rusts' strict model; i'm still interested to see what it would look like though . If i've understood right would this allow something like a '.collect()' which constructs an output collection of the same type as the input, whereas at the moment you need to 'ask for' the output type, .collect::<Vec<_> >()

[dobkeratops]

just reading through this, very interesting; according to your current thinking, (i) what would the closest approximation to haskell Functor / fmap look like, (ii) would this be able to handle abstraction over smart-pointer pointer-types too (I'm a bit hazy on where the boundaries are .. I'm not a haskell expert by any means; i'm familiar with using 'template-template parameters' in C++ and dabled with haskell a bit more since using rust. If I've understood right you seem to say this isa stepping stone to 'full hkt' that handles some popular cases ). I guess you prefer to talk about iterators rather than a clone of 'fmap' because they implement laziness/composability in rusts' strict model; i'm still interested to see what it would look like though . If i've understood right would this allow something like a '.collect()' which constructs an output collection of the same type as the input, whereas at the moment you need to 'ask for' the output type, .collect::<Vec<_> >()

[Kixunil]

Shouldn't this be type Iter<'a>: Iterator<Item = Self::Item<'a>>;? (Missing Self::)

[aturon]

I've taken the liberty of checking off the review for @pnkfelix, who is on PTO.

[aturon]

I've taken the liberty of checking off the review for @pnkfelix, who is on PTO.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period is now complete.

[rfcbot]

The final comment period is now complete.

[withoutboats]

note to t-lang: I will merge this, I have to amend it to add some unanswered questions stuff and fix a typo

[withoutboats]

note to t-lang: I will merge this, I have to amend it to add some unanswered questions stuff and fix a typo

[KodrAus]

[KodrAus]

[Ixrec]

friendly ping @withoutboats, since this sounds like it can be easily finished up and merged before the impl period

[Ixrec]

friendly ping @withoutboats, since this sounds like it can be easily finished up and merged before the impl period

[withoutboats]

Merging this finally :)

[withoutboats]

Merging this finally :)

[ashleysommer]

[ashleysommer]

[cramertj]

@withoutboats

R u tho? ;)

[cramertj]

@withoutboats

R u tho? ;)

[withoutboats]

Talked to Niko right after I posted that and found out that the thing I thought was an unresolved question was a resolved question and I have to write the section of the RFC for it, but yes

[withoutboats]

Talked to Niko right after I posted that and found out that the thing I thought was an unresolved question was a resolved question and I have to write the section of the RFC for it, but yes

[withoutboats]

Done :)

[withoutboats]

Done :)

[dylanede]

Just tried exploring how Functor would work with ATCs. Got a question about the phrase

This RFC does not propose allowing any sort of bound by the type constructor itself, whether an equality bound or a trait bound (trait bounds of course are also impossible).

Would mean that the following is not possible (note the equality bound)?

trait FunctorFamily {
    type Functor<T>;
    fn fmap<F, A, B>(x: Self::Functor<A>, f: F) -> Self::Functor<B> where F : Fn(A) -> B;
}

// The following trait is for usability
trait Functor<T> {
    type Family: FunctorFamily<Functor<T>=Self>; // equality bound
    fn fmap<F, B>(self, f: F) -> Self::Family::Functor<B> where F : Fn(Self) -> B;
}
impl<T, A> Functor<A> for T::Functor<A> where T : FunctorFamily {
    type Family = T;
    fn fmap<F, B>(self, f: F) -> T::Functor<B> where F : Fn(A) -> B {
        T::fmap(self, f)
    }
}

Is there an alternative way of phrasing FunctorFamily and Functor that works under this RFC?

[dylanede]

Just tried exploring how Functor would work with ATCs. Got a question about the phrase

This RFC does not propose allowing any sort of bound by the type constructor itself, whether an equality bound or a trait bound (trait bounds of course are also impossible).

Would mean that the following is not possible (note the equality bound)?

trait FunctorFamily {
    type Functor<T>;
    fn fmap<F, A, B>(x: Self::Functor<A>, f: F) -> Self::Functor<B> where F : Fn(A) -> B;
}

// The following trait is for usability
trait Functor<T> {
    type Family: FunctorFamily<Functor<T>=Self>; // equality bound
    fn fmap<F, B>(self, f: F) -> Self::Family::Functor<B> where F : Fn(Self) -> B;
}
impl<T, A> Functor<A> for T::Functor<A> where T : FunctorFamily {
    type Family = T;
    fn fmap<F, B>(self, f: F) -> T::Functor<B> where F : Fn(A) -> B {
        T::fmap(self, f)
    }
}

Is there an alternative way of phrasing FunctorFamily and Functor that works under this RFC?

[ExpHP]

new rendered link

[ExpHP]

new rendered link

[Boscop]

Is there already a plan how HKTs will look outside of traits, so that there is still "design space" to make the syntax the same as in traits?

How would it look in functions, like this?

fn foo<M<type, const usize>, T, const N: usize>() -> M<T, N>;

fn foo<M<type, const usize>, T, const N: usize>()
where for<T, N> M<T, N>: Default
-> M<T, N> {
    Default::default()
}

[Boscop]

Is there already a plan how HKTs will look outside of traits, so that there is still "design space" to make the syntax the same as in traits?

How would it look in functions, like this?

fn foo<M<type, const usize>, T, const N: usize>() -> M<T, N>;

fn foo<M<type, const usize>, T, const N: usize>()
where for<T, N> M<T, N>: Default
-> M<T, N> {
    Default::default()
}

[gbutler69]

It would be nice if any RFC that uses an acronym provides a link or definition upon first use. HRTB for example. So, this:

Users can bound parameters by the type constructed by that trait's associated type constructor of a trait using HRTB. Both type equality bounds and trait bounds of this kind are valid:

Would become this:

Users can bound parameters by the type constructed by that trait's associated type constructor of a trait using HRTB. Both type equality bounds and trait bounds of this kind are valid:

[gbutler69]

It would be nice if any RFC that uses an acronym provides a link or definition upon first use. HRTB for example. So, this:

Users can bound parameters by the type constructed by that trait's associated type constructor of a trait using HRTB. Both type equality bounds and trait bounds of this kind are valid:

Would become this:

Users can bound parameters by the type constructed by that trait's associated type constructor of a trait using HRTB. Both type equality bounds and trait bounds of this kind are valid:

[dylanede]

I haven't seen any mention of variance with respect to GATs yet. The RFC should note whether associated types are invariant, covariant or contravariant in their type parameters. My experiments with poor-man's GATs (for<'a> bounds and dummy traits) suggest that GATs will be severely limiting without some way of specifying variance constraints on associated types within a trait definition.

[dylanede]

I haven't seen any mention of variance with respect to GATs yet. The RFC should note whether associated types are invariant, covariant or contravariant in their type parameters. My experiments with poor-man's GATs (for<'a> bounds and dummy traits) suggest that GATs will be severely limiting without some way of specifying variance constraints on associated types within a trait definition.

[Boscop]

@dwrensha

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

How about this?
https://play.rust-lang.org/?gist=32e56d762c3ade7439cdb6796f21569f&version=stable

That's a real-world example I'm running into:

error[E0212]: cannot extract an associated type from a higher-ranked trait bound in this context
  --> src/main.rs:50:17
   |
50 |     device: Option<T::Dev>,
   |                    ^^^^^^

Does anyone of you have any idea how I can make this work? :)

I need to be able to instantiate/store AppMgr before the lifetime 'a that process() will becalled with even exists (it needs to live in an outer scope as a member of a struct, so I have to specify the type. (The 'a that process() will be called with only starts to exist shortly before calling process().)

Or is there any way I can express/enforce that DeviceApp<'a>::Dev should be the same type for all 'a? So that rustc can determine the (for<'a> DeviceApp<'a>)::Dev because there would be only one possibility for it.

---

AFAIK, to do this I'd either need the ability to express the above statement (and I'd need rustc to then allow me to use T::Dev), or I need GATs.

I tried to do it with GATs like this:

impl DeviceApp for MyAppState {
	type BorrowedState<'a> = MyBorrowedState<'a>;

but I run into this ICE:
https://play.rust-lang.org/?gist=b0614c07a45df09e1b2a3cd2bc789e7a&version=nightly

error: internal compiler error: librustc/ty/subst.rs:439: Region parameter out of range when substituting in region 'a (root type=Some(MyBorrowedState<'a>)) (index=0)
thread 'main' panicked at 'Box', librustc_errors/lib.rs:491:9
rust-lang/rust#50115

I really need this to work. What would be the recommended way to do this before this ICE issue gets fixed? :)

Btw, how long do you think it will take until this issue gets fixed?

[Boscop]

@dwrensha

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

How about this?
https://play.rust-lang.org/?gist=32e56d762c3ade7439cdb6796f21569f&version=stable

That's a real-world example I'm running into:

error[E0212]: cannot extract an associated type from a higher-ranked trait bound in this context
  --> src/main.rs:50:17
   |
50 |     device: Option<T::Dev>,
   |                    ^^^^^^

Does anyone of you have any idea how I can make this work? :)

I need to be able to instantiate/store AppMgr before the lifetime 'a that process() will becalled with even exists (it needs to live in an outer scope as a member of a struct, so I have to specify the type. (The 'a that process() will be called with only starts to exist shortly before calling process().)

Or is there any way I can express/enforce that DeviceApp<'a>::Dev should be the same type for all 'a? So that rustc can determine the (for<'a> DeviceApp<'a>)::Dev because there would be only one possibility for it.

---

AFAIK, to do this I'd either need the ability to express the above statement (and I'd need rustc to then allow me to use T::Dev), or I need GATs.

I tried to do it with GATs like this:

impl DeviceApp for MyAppState {
	type BorrowedState<'a> = MyBorrowedState<'a>;

but I run into this ICE:
https://play.rust-lang.org/?gist=b0614c07a45df09e1b2a3cd2bc789e7a&version=nightly

error: internal compiler error: librustc/ty/subst.rs:439: Region parameter out of range when substituting in region 'a (root type=Some(MyBorrowedState<'a>)) (index=0)
thread 'main' panicked at 'Box', librustc_errors/lib.rs:491:9
rust-lang/rust#50115

I really need this to work. What would be the recommended way to do this before this ICE issue gets fixed? :)

Btw, how long do you think it will take until this issue gets fixed?

[Boscop]

I found a workaround, pulling Dev outside of the <'a> quantification scope:

https://play.rust-lang.org/?gist=9b7907d94dfc28aabe12f5f90f8c4592&version=stable

pub trait DeviceSpecific {
	type Dev: Device;
}

pub trait DeviceApp<'a>: DeviceSpecific {
// 	type Dev: Device;

This compiles, but it's a combo of 2 ugly workarounds (I'd prefer not to split the trait like that) that wouldn't be necessary with proper (working) GATs, right?

Can you think of any solution that doesn't require me to split the trait like that? :)

[Boscop]

I found a workaround, pulling Dev outside of the <'a> quantification scope:

https://play.rust-lang.org/?gist=9b7907d94dfc28aabe12f5f90f8c4592&version=stable

pub trait DeviceSpecific {
	type Dev: Device;
}

pub trait DeviceApp<'a>: DeviceSpecific {
// 	type Dev: Device;

This compiles, but it's a combo of 2 ugly workarounds (I'd prefer not to split the trait like that) that wouldn't be necessary with proper (working) GATs, right?

Can you think of any solution that doesn't require me to split the trait like that? :)

[Boscop]

Ah no, it doesn't work with the impl of AppMgr :(

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	device: Option<T::Dev>,
	check_connection: AtFps,
	in_port_name: String,
	out_port_name: String,
	app: T,
}

impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {

	// error: cannot extract an associated type from a higher-ranked trait bound in this context

	pub fn process(&mut self, now: u64, v: &mut T::BorrowedState) -> Vec<T::Event> {
	//                                          ^^^^^^^^^^^^^^^^
		if self.check_connection.is_due(now) {
			let should_reconnect = if let Some(ref mut device) = self.device {
				!device.is_connected()
			} else {
				true
			};
			if should_reconnect {
				self.device = T::Dev::connect(&self.in_port_name, &self.out_port_name).ok();
			}
		}
		if let Some(ref mut device) = self.device {
			let app = &mut self.app;
			device.frame(now, |device, input| { app.process(device, input, v) })
		} else {
			vec![]
		}
	}
}

What I need is this:

// pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
pub struct AppMgr<T: DeviceApp> { // DeviceApp not depending on the lifetime 'a that process() gets called with
	// ...
}

// impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {
impl<T: DeviceApp> AppMgr<T> {
	// instantiating T::BorrowedState with <'a> from the scope that process() is called in
	pub fn process<'a>(&mut self, now: u64, v: &'a mut T::BorrowedState<'a>) -> Vec<T::Event> {
		// ...
	}
}

Is there any way to do this now? If not, will this work with proper working GATs?

[Boscop]

Ah no, it doesn't work with the impl of AppMgr :(

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	device: Option<T::Dev>,
	check_connection: AtFps,
	in_port_name: String,
	out_port_name: String,
	app: T,
}

impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {

	// error: cannot extract an associated type from a higher-ranked trait bound in this context

	pub fn process(&mut self, now: u64, v: &mut T::BorrowedState) -> Vec<T::Event> {
	//                                          ^^^^^^^^^^^^^^^^
		if self.check_connection.is_due(now) {
			let should_reconnect = if let Some(ref mut device) = self.device {
				!device.is_connected()
			} else {
				true
			};
			if should_reconnect {
				self.device = T::Dev::connect(&self.in_port_name, &self.out_port_name).ok();
			}
		}
		if let Some(ref mut device) = self.device {
			let app = &mut self.app;
			device.frame(now, |device, input| { app.process(device, input, v) })
		} else {
			vec![]
		}
	}
}

What I need is this:

// pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
pub struct AppMgr<T: DeviceApp> { // DeviceApp not depending on the lifetime 'a that process() gets called with
	// ...
}

// impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {
impl<T: DeviceApp> AppMgr<T> {
	// instantiating T::BorrowedState with <'a> from the scope that process() is called in
	pub fn process<'a>(&mut self, now: u64, v: &'a mut T::BorrowedState<'a>) -> Vec<T::Event> {
		// ...
	}
}

Is there any way to do this now? If not, will this work with proper working GATs?

[Boscop]

I found a way to impl AppMgr that works:

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	device: Option<T::Dev>,
	// ...
}

impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {
	pub fn process<'a>(&mut self, now: u64, msgs: Vec<<T as DeviceApp<'a>>::Msg>, v: &'a mut <T as DeviceApp<'a>>::BorrowedState) -> Vec<<T as DeviceApp<'a>>::Event> {
		// ...

Here as reduced working example:
https://play.rust-lang.org/?gist=7522f287efe68c7b73efcddef1c5dd2a&version=stable

But I still wasn't happy about having to split the trait, so I thought, if I mandate that Dev: 'static, the compiler will be able to infer that for all 'a, <T as DeviceApp<'a>>::Dev will be the same, namely the same as <T as DeviceApp<'static>>::Dev (because Dev: 'static means it won't depend on 'a).
So I tried that, bringing Dev back into the DeviceApp trait:

/*
pub trait DeviceSpecific {
	type Dev: Device;
}

pub trait DeviceApp<'a>: DeviceSpecific {
*/
pub trait DeviceApp<'a> {
	type Dev: Device + 'static;

// ...

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	// device: Option<T::Dev>,
	device: Option<<T as DeviceApp<'static>>::Dev>, // because Dev: 'static, i thought it will be unifyable with any 'a
	app: T,
	// ...
}

But now I get this error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
  --> src/main.rs:79:53
   |
79 |             device.frame(now, |device, input| { app.process(device, input, v) })
   |                                                     ^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime 'a as defined on the method body at 75:5...
  --> src/main.rs:75:5
   |
75 |     pub fn process<'a>(&mut self, now: u64, msgs: Vec<<T as DeviceApp<'a>>::Msg>, v: &'a mut <T as DeviceApp<'a>>::BorrowedState) -> Vec<<T as DeviceApp<'a>>::Event> {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: ...so that the expression is assignable:
           expected &mut <T as DeviceApp<'_>>::BorrowedState
              found &mut <T as DeviceApp<'a>>::BorrowedState
   = note: but, the lifetime must be valid for the static lifetime...
   = note: ...so that the expression is assignable:
           expected &mut <T as DeviceApp<'_>>::Dev
              found &mut <T as DeviceApp<'static>>::Dev

https://play.rust-lang.org/?gist=d4084c0a87890d4ed17d0491e453565b&version=stable

Why can't rustc unify <T as DeviceApp<'static>>::Dev> with <T as DeviceApp<'a>>::Dev> for all 'a, when it KNOWS that Dev: 'static (the type doesn't depend on 'a, so it can only be the same for all 'a)?

---

Actually, for some reason, in my real code I get a different (similar) error instead (with nightly 2018-03-06):

error: free region `'a` does not outlive free region `'static`
   --> src\devices\mod.rs:132:15
    |
132 |         if let Some(ref mut device) = self.device {
    |                     ^^^^^^^^^^^^^^

Any idea how to make it work with Dev inside the DeviceApp trait? :)

[Boscop]

I found a way to impl AppMgr that works:

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	device: Option<T::Dev>,
	// ...
}

impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {
	pub fn process<'a>(&mut self, now: u64, msgs: Vec<<T as DeviceApp<'a>>::Msg>, v: &'a mut <T as DeviceApp<'a>>::BorrowedState) -> Vec<<T as DeviceApp<'a>>::Event> {
		// ...

Here as reduced working example:
https://play.rust-lang.org/?gist=7522f287efe68c7b73efcddef1c5dd2a&version=stable

But I still wasn't happy about having to split the trait, so I thought, if I mandate that Dev: 'static, the compiler will be able to infer that for all 'a, <T as DeviceApp<'a>>::Dev will be the same, namely the same as <T as DeviceApp<'static>>::Dev (because Dev: 'static means it won't depend on 'a).
So I tried that, bringing Dev back into the DeviceApp trait:

/*
pub trait DeviceSpecific {
	type Dev: Device;
}

pub trait DeviceApp<'a>: DeviceSpecific {
*/
pub trait DeviceApp<'a> {
	type Dev: Device + 'static;

// ...

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	// device: Option<T::Dev>,
	device: Option<<T as DeviceApp<'static>>::Dev>, // because Dev: 'static, i thought it will be unifyable with any 'a
	app: T,
	// ...
}

But now I get this error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
  --> src/main.rs:79:53
   |
79 |             device.frame(now, |device, input| { app.process(device, input, v) })
   |                                                     ^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime 'a as defined on the method body at 75:5...
  --> src/main.rs:75:5
   |
75 |     pub fn process<'a>(&mut self, now: u64, msgs: Vec<<T as DeviceApp<'a>>::Msg>, v: &'a mut <T as DeviceApp<'a>>::BorrowedState) -> Vec<<T as DeviceApp<'a>>::Event> {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: ...so that the expression is assignable:
           expected &mut <T as DeviceApp<'_>>::BorrowedState
              found &mut <T as DeviceApp<'a>>::BorrowedState
   = note: but, the lifetime must be valid for the static lifetime...
   = note: ...so that the expression is assignable:
           expected &mut <T as DeviceApp<'_>>::Dev
              found &mut <T as DeviceApp<'static>>::Dev

https://play.rust-lang.org/?gist=d4084c0a87890d4ed17d0491e453565b&version=stable

Why can't rustc unify <T as DeviceApp<'static>>::Dev> with <T as DeviceApp<'a>>::Dev> for all 'a, when it KNOWS that Dev: 'static (the type doesn't depend on 'a, so it can only be the same for all 'a)?

---

Actually, for some reason, in my real code I get a different (similar) error instead (with nightly 2018-03-06):

error: free region `'a` does not outlive free region `'static`
   --> src\devices\mod.rs:132:15
    |
132 |         if let Some(ref mut device) = self.device {
    |                     ^^^^^^^^^^^^^^

Any idea how to make it work with Dev inside the DeviceApp trait? :)

[burdges]

I quite like the StreamingIterator example here, but another question: Should we permit higher kinded type parameters? As an example, you could borrow a data structure and use mut reborrows to iterate over it several times like:

fn process_elements<V: ?Sized,II>(x &mut V,f: fn<'a>(&mut V) -> II)
where
    II<'a>: Iterator<Item=&mut X>

or

fn process_elements<F,II>(f: &mut F)
where 
    for<'a> &'a mut F: Into<II<a'>>,
    for<'a> II<'a>: Iterator<Item=&'a mut X>

In principle, these could be done with impl Trait in certain positions, but this feature seems logically prior.

[burdges]

I quite like the StreamingIterator example here, but another question: Should we permit higher kinded type parameters? As an example, you could borrow a data structure and use mut reborrows to iterate over it several times like:

fn process_elements<V: ?Sized,II>(x &mut V,f: fn<'a>(&mut V) -> II)
where
    II<'a>: Iterator<Item=&mut X>

or

fn process_elements<F,II>(f: &mut F)
where 
    for<'a> &'a mut F: Into<II<a'>>,
    for<'a> II<'a>: Iterator<Item=&'a mut X>

In principle, these could be done with impl Trait in certain positions, but this feature seems logically prior.