Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic associated types (associated type constructors) #1598

Merged
merged 10 commits into from Sep 2, 2017

Conversation

@withoutboats
Copy link
Contributor

withoutboats commented Apr 30, 2016

I definitely realize that this feature may not be prioritized right now, and I expect that some aspects of it probably present real implementation challenges (especially partial application of type operators and extending HRTBs), but I wanted to document and discuss what seems to me like the best incremental step toward higher-kinded polymorphism in Rust.

Rendered

}
```

`not has the type `bool -> bool` (my apologies for using a syntax different

This comment has been minimized.

Copy link
@kennytm

kennytm Apr 30, 2016

Member

There is a missing ` after the "not".

```

`&[u8]` is not an elided form of some `&'a [u8]`, but a type operator of the
kind `& -> *`.

This comment has been minimized.

Copy link
@P1start

P1start Apr 30, 2016

Contributor

This syntax feels a bit too magic to me—&[u8] almost always acts as &'a [u8] with 'a elided, and it’s not immediately obvious at a glance that Item is actually a higher-order type of kind & -> *. Also, what if multiple lifetime parameters are omitted? Is &&T equivalent to <'a, 'b> => &'a &'b T or <'a, 'b> => &'b &'a T?

I would prefer a more obvious (but more verbose) syntax: StreamingIterator<Item<'a>=&'a [u8]>. That way it’s apparent that 'a is declared as part of the trait bound, and it also nicely matches the syntax for associated item definitions.

This comment has been minimized.

Copy link
@glaebhoerl

glaebhoerl Apr 30, 2016

Contributor

That also works better if the associated type is generic over a type rather than a lifetime -- Foo<Bar<T>=&'static [T]> is certainly more obvious than Foo<Bar=&'static []>, for example.

This comment has been minimized.

Copy link
@arielb1

arielb1 Apr 30, 2016

Contributor

It certainly fits the pattern of lifetime elision done today - fn(&[u8]) is today for<'a> fn(&'a [u8]), so Stream<Item=&[u8]> is Stream<Item=λ'a. &'a [u8]>. Of course we probably want a fully-explicit way of writing these things.

By the way, I would not like to use & to refer to lifetime - it has a perfectly good meaning of reference today.

This comment has been minimized.

Copy link
@eddyb

eddyb Apr 30, 2016

Member

By the way, I would not like to use & to refer to lifetime - it has a perfectly good meaning of reference today.

I agree, why not '*?

This comment has been minimized.

Copy link
@krdln

krdln Apr 30, 2016

Contributor

The "elision" syntax (I mean StreamingIterator<Item=&[u8]>) makes sense for '* → * (& → *) types and I don't think it would create confusion or ambiguity. But for bigger number of input params or non-lifetime inputs I can't image how it would work, so we need a non-elided syntax too. I thought of reusing the closure syntax (Item = |'a| &'a[u8]), but the one proposed by @P1start (Item<'a>=&'a [u8]) looks more Rusty imho.

@withoutboats You are talking about allowing any operators, but the syntax seems only defined for constructors. If you plan to support any operators, could you give an example of using * → * → * associated operator (not (*, *) → *)? Also, an example for type as an input would be nice too. I know it's analogous to lifetime as an input, but it would be nice to have it for completeness. An example would be also nice for multi-input constructor.

This comment has been minimized.

Copy link
@eddyb

eddyb Apr 30, 2016

Member

@krdln The "type-level closure syntax" already exists, and the example would be Item = for<'a> &'a [u8], although I prefer Item<'a> = &'a [u8] myself.

This comment has been minimized.

Copy link
@krdln

krdln Apr 30, 2016

Contributor

@eddyb The (implements) for<'a> 'a[u8] means ∀'a (implements) &'a[u8]. The (implements) for<'a> Trait<'a> means ∀'a (implements) Trait<'a>. I think it's kind-of analogous, but a totally different thing than a type-lambda. And it would be confusing to use the same syntax for those two things.

If you squint, you can imagine that for<'a> Trait<'a> has a type signature of (& → * → bool) → (* → bool). Which means "take a lifetime parametrised (& → ...) trait (* → bool) and return a trait (* → bool)" * → bool (a trait bound). And that's a totally different thing than & → *.

This comment has been minimized.

Copy link
@eddyb

eddyb Apr 30, 2016

Member

T: for<'a> Trait<'a> is sugar for for<'a> T: Trait<'a>, i.e. the universal quantification covers the whole bound.
I suppose it would be a bit weird if the following two snippets were equivalent:

type Foo = for<'a> fn(&'a T) -> &'a U;
type Bar = for<'a> Trait<'a>;
type Foo<'a> = fn(&'a T) -> &'a U;
type Bar<'a> = Trait<'a>;

And if omitting explicit <'x> would mean universal quantification, which conflicts with anonymous lifetimes in a function signature and/or elision.
Although, this is only a problem with lifetimes AFAICT, unless there are situations where you can actually have a Box<for<T: Foo> Trait<T>>.

This comment has been minimized.

Copy link
@withoutboats

withoutboats Apr 30, 2016

Author Contributor

I would prefer a more obvious (but more verbose) syntax: StreamingIterator<Item<'a>=&'a [u8]>. That way it’s apparent that 'a is declared as part of the trait bound, and it also nicely matches the syntax for associated item definitions.

I agree that this syntax is quite magic (and is easily the least satisfying syntax here to me), but I'm concerned about how much this looks like some sort of higher-rank syntax. If we do add an ability to make these parameters higher-rank, I am a bit concerned about the difference in meaning about 'a between Item<'a>=&'a [u8] and for<'a> Item<'a>: Display.

But for bigger number of input params or non-lifetime inputs I can't image how it would work, so we need a non-elided syntax too.

You're right, the elision syntax is right out for that reason.

By the way, I would not like to use & to refer to lifetime - it has a perfectly good meaning of reference today.

This syntax is not internal to Rust of course. I find '* hard to distinguish from *. I think the best syntax might be the clearest, using words: lifetime -> type.

This comment has been minimized.

Copy link
@glaebhoerl

glaebhoerl Apr 30, 2016

Contributor

I think the best syntax might be the clearest, using words: lifetime -> type.

I agree!

Not even Haskellers really like their * syntax for "type" and in GHC 8 they're getting Type as an alias for it.

@glaebhoerl
Copy link
Contributor

glaebhoerl commented Apr 30, 2016

I agree that this seems like the ideal starting point for adding higher-kinded capability to Rust. Even the idea of moving forward incrementally in this way, as opposed to "Adding Higher-Kinded Types" all at once in a "big bang", is something that hadn't occurred to me, but seems obvious in hindsight.

To stick with our own terminology consistently, instead having of a mishmash of vocabulary from Rust, Haskell, and the academic literature, I would probably prefer to call this feature "generic associated types" or "associated generic types". (I would also prefer to call "higher-kinded types" instead "higher-order generic types", contrasting with "higher-rank types" which are "higher-order generic functions", but the usage of "HKT" is incredibly widespread by this point.)

I also suspect that most of the difficult design work which needs to be done before this feature can move forward is on how to extend the type system to support it, which this RFC (self-admittedly) doesn't even attempt to address. Am I right?

@dwrensha
Copy link

dwrensha commented Apr 30, 2016

Currently, an approximation of StreamingIterator can be defined like this:

trait StreamingIterator<'a> {
  type Item: 'a;
  fn next(&'a self) -> Option<Self::Item>;
}

You can then bound types as T: for<'a> StreamingIterator<'a> to avoid the lifetime parameter infecting everything StreamingIterator appears.

However, this only partially prevents the infectiveness of StreamingIterator, only allows for some of the types that associated type operators can express, and is in generally a hacky attempt to work around the limitation rather than an equivalent alternative.

I'm interested in seeing a concrete example of something that cannot be expressed using this hack. I've hit situations where I've wanted associated type operators as proposed in this RFC, but in every case it has sufficed hoist the the type parameters up to the trait itself. See, for example, these capnp traits. I agree that such hoisting can be ugly, and I feel like there are probably cases where it fails, but those cases are not immediately obvious to me.

@BurntSushi
Copy link
Member

BurntSushi commented Apr 30, 2016

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

Ideally, the StreamingIterator trait should be expanded to contain many of the same utility adapter methods that the Iterator trait has.

@eddyb
Copy link
Member

eddyb commented Apr 30, 2016

AFAIK the bulk of the implementation work needed here is removing the constraints on the "3 application levels" in the current compiler (type/trait parameters, Self and fn/method parameters).

The actual resolution of non-monomorphic associated types won't actually be that big of a problem, we already have to do that sort of transformation for generic methods.

And lastly, what I'm worried most about is checking uses of such a trait with polymorphic associated types without providing concrete bindings.
We've had a few stages of adjustment to all the problems associated types can cause in various edge cases.
I suspect this is the "difficult design work" @glaebhoerl was talking about, in which case I agree.

But overall, I like this RFC and @rust-lang/compiler will continue to work towards making such advances possible, regardless of which exact syntactic or semantic choices we end up accepting.

@krdln
Copy link
Contributor

krdln commented Apr 30, 2016

@dwrensha
I think that it's currently impossible to express something like owning_ref, but generic over reference type. Eg. the interface of generic reference could look like:

unsafe trait Ref {
    type Item<'a>;
    unsafe fn erase_lifetime<'a>(x: Item<'a>) -> Item<'static> { transmute(x) }
    unsafe fn introduce_lifetime<'a>(x: Item<'static>) -> Item<'a> { transmute(x) }
}

The main benefit from this RFC here would be that compiler would know that all types created by the same & → * constructor are transmute-compatible, and that it can apply that constructor for any lifetime. Unfortunately, the trait would have to be implemented for some phantom type, since this RFC doesn't allow traits to be implemented for type constructors, but I think it's only a minor issue.

It seems that the main benefit from this RFC comes from allowing lifetime (& → *) constructors, so I'd be happy to see it accepted even if it doesn't allow full-featured * → * ones. (analogicaly, as we have for<'a> syntax only for lifetimes)

@eddyb
Copy link
Member

eddyb commented Apr 30, 2016

@krdln Actually, right now you can't even use transmute there, although I would be inclined to accept a same-type exception (we already erase lifetimes, so it should just see Item<'static> on both sides).

@krdln
Copy link
Contributor

krdln commented Apr 30, 2016

@eddyb Your example doesn't compile, since you can't apply an lifetime paremeter to Self::Item. Maybe you've meant to use Ref<'a> somewhere there? I've tried, but I couldn't make it compile. So I'm still not convinced that thing is possible in today's Rust (but I won't be surprised if it was possible, Rust can be really surprising).

Using your suggestion, I came up with this, so this use case might be supported in current Rust, with some fixes for transmute behaviour.

@eddyb
Copy link
Member

eddyb commented Apr 30, 2016

@krdln Hah, that was dumb of me. Totally missed the fact that you were applying multiple times within the trait itself. It could work via free functions with R: Ref<'a> + Ref<'static> but that's probably not useful to you.

@soltanmm
Copy link

soltanmm commented Apr 30, 2016

@krdln I think there does exist an injection between what is proposed here and the syntax that exists today, but it requires a few more pieces than what @dwrensha pointed out.

use std::mem;

// '* -> *
trait R2T<'a> {
    type T;
}

// * -> *
trait T2T<A> {
    type T;
}

// ('* -> *)('*)
type AppR<'a, T> where T: R2T<'a> = <T as R2T<'a>>::T;
// (* -> *)(*)
type AppT<A, T> where T: T2T<A> = <T as T2T<A>>::T;

// A trait with a higher kinded operator
unsafe trait Ref {
    type Item: for<'a> R2T<'a>;
    unsafe fn erase_lifetime<'a>(x: AppR<'a, Self::Item>) -> AppR<'static, Self::Item> { mem::transmute(x) }
    unsafe fn introduce_lifetime<'a>(x: AppR<'static, Self::Item>) -> AppR<'a, Self::Item> { mem::transmute(x) }
}

// A type with a '* -> '* -> * HKT
trait MyType
    where
        for<'a> Self::BiRefItem: R2T<'a>,
        for<'a, 'b> AppR<'a, Self::BiRefItem>: R2T<'b>
{
    type BiRefItem;
}

// Use of partial application by currying (which can always be extended to full partial application with helpers)
trait PartiallyApplied<'a, T: MyType>
    where
        Self::Ref: Ref<Item=AppR<'a, T::BiRefItem>>
{
    type Ref;
    // ...
}

fn main() {}

However, it is excessively verbose, highly error prone, works only for lifetimes in certain use-cases (because it depends on HRTBs), and there're some issues with the compiler that prevent it from compiling (see rust-lang/rust#30472 and rust-lang/rust#31580).

For a more practical application of the general approach that does work today, see #1403 (comment).

For background on the general approach, see the fine-print in the associated items RFC. n.b. the RFC uses non-existent syntax in its example.

and I am in no way suggesting that one take a syntax-sugar approach implementing this :-P

@soltanmm
Copy link

soltanmm commented Apr 30, 2016

Actually, now that I think about it... Unless this RFC is limited to kinds that look like '* -> *, doesn't it require extending HRTBs to types (in mechanism, not necessarily in syntax) to be useful? Otherwise we can't put bounds on the outputs of HKTs like * -> *. Seems like doing that first could be a reasonable alternative.

@eddyb
Copy link
Member

eddyb commented Apr 30, 2016

@soltanmm It is limited to '* -> *. However, this brings an interesting point I didn't realize: T: Trait<Item<'a> = &'a [u8]> is sugar for T: Trait and for<'a> <T as Trait>::Item<'a> = &'a [u8] so still HRTB in nature, just packaged away neatly.

@withoutboats
Copy link
Contributor Author

withoutboats commented Apr 30, 2016

To stick with our own terminology consistently, instead having of a mishmash of vocabulary from Rust, Haskell, and the academic literature, I would probably prefer to call this feature "generic associated types" or "associated generic types". (I would also prefer to call "higher-kinded types" instead "higher-order generic types", contrasting with "higher-rank types" which are "higher-order generic functions", but the usage of "HKT" is incredibly widespread by this point.)

I don't feel strongly about the terminology, but I'll note that I avoided the term "higher-kinded type" in this RFC because I found it very confusing before I read TaPL, which avoids that terminology. A big part of my confusion was that calling type operators "types" suggests to me that this is a language feature that allows one to create values of a "type" of a higher kinded, like let x: Vec.

I also don't think we use the term generics that much, do we? I usually see 'type parameter' or 'parameterized type.'

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

I think others have answered this well (the most obvious thing is: * -> *), but I want to add that the real point is to make this feature easier and more intuitive to use, just as adding const parameters would be an easier easier and more intuitive feature than the type-level peano number hacks that we have right now.

You are talking about allowing any operators, but the syntax seems only defined for constructors. If you plan to support any operators, could you give an example of using * → * → * associated operator (not (*, *) → *)?

I used the term operator throughout the text, but its correct that this would only allow associated type constructors.

Actually, now that I think about it... Unless this RFC is limited to kinds that look like '* -> *, doesn't it require extending HRTBs to types (in mechanism, not necessarily in syntax) to be useful? Otherwise we can't put bounds on the outputs of HKTs like * -> *. Seems like doing that first could be a reasonable alternative.

I don't know anything about how HRTBs are implemented, some of eddyb's comments suggest this might be an easy extension, but the higher-ranked parameter is being applied in a different position in these two expressions:

for<'a> Type<'a>: Trait
Type: for<'a> Trait<'a>

This RFC explicitly says it doesn't propose allowing for<'a> Type<'a>: Trait, though obviously that would be the most useful thing to add next. I this RFC is useful even if you can't bound this new kind of associated item.

Also, this RFC only discusses lifetime -> type associated items, but I don't think there's anything in the RFC that doesn't apply equivalently to all kinds of type constructor. The main reason lifetime -> type is the only kind discussed is its the only kind for which I know of obvious, valuable use cases. Not coincidentally, I think its also the kind which can be easily explained without type theory concepts: it lets the trait define "view" types for its receiver.

Extending HRTB to be used with associated items of a kind other than lifetime -> type would require more work, though, of course.

@withoutboats
Copy link
Contributor Author

withoutboats commented Apr 30, 2016

Thinking about the syntactic and semantic complexities of bounds using associated type constructors, and the fact that all of this seems to be conceivable as just an extension of HRTBs, I think it might be a good idea to not even discuss bounds using associated type constructors in this RFC, and instead delegate that to an RFC about extending HRTBs.

@eddyb
Copy link
Member

eddyb commented Apr 30, 2016

@withoutboats Both of those forms are allowed right now and the for<'a> actually gets lifted to the whole bound (if you don't mean (for<'a> Type<'a>): Trait, of course).

In a way, for<'a> is only syntactical, at the semantic level debruijn indices are used for "late-bound lifetimes" with "binders" which define the boundaries where they apply.

So for<'a> (for<'b> fn(&'b T) -> (&'b U, &'a V)): Trait<'a> ends up being something like fn(&'2 T) -> (&'2 U, &'1 V): Trait<'1> with a binder around the signature of the fn type and another one around the whole bound.

it might be a good idea to not even discuss bounds using associated type constructors in this RFC

It's not a good idea to punt on this matter, because the T: Trait<Assoc = U> syntax is sugar for a <T as Trait>::Assoc = U bound (which we can't express right now syntactically, sadly).

@withoutboats
Copy link
Contributor Author

withoutboats commented Apr 30, 2016

It's not a good idea to punt on this matter, because the T: Trait<Assoc = U> syntax is sugar for a <T as Trait>::Assoc = U bound (which we can't express right now syntactically, sadly).

I'm not sure why this makes it bad to punt, but I think the important takeaway is that we (yet) don't need to be able to put type constructors in where clauses, because we can use HRTB. Obviously this won't be sufficient for later use cases involving higher-kinded traits.

@withoutboats
Copy link
Contributor Author

withoutboats commented Apr 30, 2016

Another issue has occurred to me just now. We would probably also want some sort of implicit HRTBs at the declaration site of the associated type, as in:

trait Collection {
    type Elem;
    type Iter<'a>: Iterator<Item=&'a Self::Elem>;
    type IterMut<'a>: Iterator<Item=&'a mut Self::Elem>;
    ...
}
@eddyb
Copy link
Member

eddyb commented Apr 30, 2016

@withoutboats I wonder if we should agree on using "polymorphic" or some other "painfully clear" term because I thought you meant <T as Trait>::PolyAssoc<'a>: RegularTrait instead of <T as Trait>::PolyAssoc: SomethingNew, and the latter is HKT territory as far as I'm concerned, so we're in violent agreement, I suppose.

@withoutboats
Copy link
Contributor Author

withoutboats commented Apr 30, 2016

@eddyb: Right, the RFC currently discusses <T as Trait>::PolyAssoc=TypeConstructor, but instead of allowing that that we can just use for<'a> <T as Trait>::PolyAssoc<'a>=TypeConstructor<'a>

@withoutboats
Copy link
Contributor Author

withoutboats commented May 2, 2016

I've edited and expanded the RFC in response to the comments so far, mainly clarifications and adding content about how this would require HRTB changes.

@withoutboats withoutboats force-pushed the withoutboats:associated_hkts branch 2 times, most recently from 188eaeb to c73b60e May 2, 2016
@nrc nrc added the T-lang label May 2, 2016
@withoutboats withoutboats changed the title Associated type operators (a form of higher-kinded polymorphism). Associated type constructors (a form of higher-kinded polymorphism). May 2, 2016
1. First, allowing HRTBs to applied to type constructors on the left-hand side
of the bound. Currently they are only available to traits on the right-hand
side of the bound.
2. Second, allowing HRTBs to introduce type parameters, instead of only

This comment has been minimized.

Copy link
@soltanmm

soltanmm May 2, 2016

When's the RFC to spec out the syntax of bounds/where-clauses on HRTB-quantified types coming out? 😃

aaand obligatory cc #1481

Enabling this requires extending HRTBs in two different ways:

1. First, allowing HRTBs to applied to type constructors on the left-hand side
of the bound. Currently they are only available to traits on the right-hand

This comment has been minimized.

Copy link
@soltanmm

soltanmm May 2, 2016

okay github hopefully you won't just eat my comment this time...
I thought we determined otherwise. Or do you mean something outside the form of for<'x> T: U? If something different it's probably worth clarifying.

This comment has been minimized.

Copy link
@withoutboats

withoutboats May 2, 2016

Author Contributor

You're right, I misunderstood and thought that it was disallowed syntactically.

@eefriedman
Copy link
Contributor

eefriedman commented May 2, 2016

The syntax struct Foo<T> where T: for<'a> StreamingIterator<Item=&'a mut [u8]> from the example in the RFC doesn't seem right. The following expanded example is legal on nightly Rust:

trait StreamingIterator {
    type Item;
}
struct Foo<T> where T: for<'a> StreamingIterator<Item=&'a mut [u8]> {
    x: T
}

Granted, as far as I can tell there is no way to write T which satisfies this constraint at the moment, but as far as I can tell that's orthogonal to this RFC.

I think what you actually want is something closer to struct Foo<T> where T: StreamingIterator<Item<'a>=&'a mut [u8]>, with no for<>.

@withoutboats
Copy link
Contributor Author

withoutboats commented May 2, 2016

You're right that its wrong, but it needs both the <'a> and the for<'a>. Without the for<'a>, it would search the surrounding scope for a lifetime parameter called 'a and fail.

EDIT: Fixed.

@withoutboats withoutboats force-pushed the withoutboats:associated_hkts branch from c73b60e to 8e922c0 May 2, 2016
@withoutboats
Copy link
Contributor Author

withoutboats commented Sep 2, 2017

Talked to Niko right after I posted that and found out that the thing I thought was an unresolved question was a resolved question and I have to write the section of the RFC for it, but yes

@withoutboats withoutboats merged commit a7cd910 into rust-lang:master Sep 2, 2017
@withoutboats
Copy link
Contributor Author

withoutboats commented Sep 2, 2017

Done :)

@dylanede
Copy link

dylanede commented Sep 8, 2017

Just tried exploring how Functor would work with ATCs. Got a question about the phrase

This RFC does not propose allowing any sort of bound by the type constructor itself, whether an equality bound or a trait bound (trait bounds of course are also impossible).

Would mean that the following is not possible (note the equality bound)?

trait FunctorFamily {
    type Functor<T>;
    fn fmap<F, A, B>(x: Self::Functor<A>, f: F) -> Self::Functor<B> where F : Fn(A) -> B;
}

// The following trait is for usability
trait Functor<T> {
    type Family: FunctorFamily<Functor<T>=Self>; // equality bound
    fn fmap<F, B>(self, f: F) -> Self::Family::Functor<B> where F : Fn(Self) -> B;
}
impl<T, A> Functor<A> for T::Functor<A> where T : FunctorFamily {
    type Family = T;
    fn fmap<F, B>(self, f: F) -> T::Functor<B> where F : Fn(A) -> B {
        T::fmap(self, f)
    }
}

Is there an alternative way of phrasing FunctorFamily and Functor that works under this RFC?

jgouly added a commit to jgouly/keyboard-app that referenced this pull request Oct 6, 2017
This is needed so that get_row_pin/get_column_pin's return value can borrow
from &self.

This could be solved more ergonomically using a HKT constructor, as defined
in this Rust RFC: rust-lang/rfcs#1598
@ExpHP
Copy link

ExpHP commented Oct 17, 2017

@samsartor samsartor mentioned this pull request Oct 30, 2017
@Boscop
Copy link

Boscop commented Nov 3, 2017

Is there already a plan how HKTs will look outside of traits, so that there is still "design space" to make the syntax the same as in traits?

How would it look in functions, like this?

fn foo<M<type, const usize>, T, const N: usize>() -> M<T, N>;

fn foo<M<type, const usize>, T, const N: usize>()
where for<T, N> M<T, N>: Default
-> M<T, N> {
    Default::default()
}
@gbutler69
Copy link

gbutler69 commented Mar 11, 2018

It would be nice if any RFC that uses an acronym provides a link or definition upon first use. HRTB for example. So, this:

Users can bound parameters by the type constructed by that trait's associated type constructor of a trait using HRTB. Both type equality bounds and trait bounds of this kind are valid:

Would become this:

Users can bound parameters by the type constructed by that trait's associated type constructor of a trait using HRTB. Both type equality bounds and trait bounds of this kind are valid:

@dylanede
Copy link

dylanede commented Apr 16, 2018

I haven't seen any mention of variance with respect to GATs yet. The RFC should note whether associated types are invariant, covariant or contravariant in their type parameters. My experiments with poor-man's GATs (for<'a> bounds and dummy traits) suggest that GATs will be severely limiting without some way of specifying variance constraints on associated types within a trait definition.

@Boscop
Copy link

Boscop commented Apr 21, 2018

@dwrensha

I'm interested in seeing a concrete example of something that cannot be expressed using this hack.

How about this?
https://play.rust-lang.org/?gist=32e56d762c3ade7439cdb6796f21569f&version=stable

That's a real-world example I'm running into:

error[E0212]: cannot extract an associated type from a higher-ranked trait bound in this context
  --> src/main.rs:50:17
   |
50 |     device: Option<T::Dev>,
   |                    ^^^^^^

Does anyone of you have any idea how I can make this work? :)

I need to be able to instantiate/store AppMgr before the lifetime 'a that process() will becalled with even exists (it needs to live in an outer scope as a member of a struct, so I have to specify the type. (The 'a that process() will be called with only starts to exist shortly before calling process().)

Or is there any way I can express/enforce that DeviceApp<'a>::Dev should be the same type for all 'a? So that rustc can determine the (for<'a> DeviceApp<'a>)::Dev because there would be only one possibility for it.


AFAIK, to do this I'd either need the ability to express the above statement (and I'd need rustc to then allow me to use T::Dev), or I need GATs.

I tried to do it with GATs like this:

impl DeviceApp for MyAppState {
	type BorrowedState<'a> = MyBorrowedState<'a>;

but I run into this ICE:
https://play.rust-lang.org/?gist=b0614c07a45df09e1b2a3cd2bc789e7a&version=nightly

error: internal compiler error: librustc/ty/subst.rs:439: Region parameter out of range when substituting in region 'a (root type=Some(MyBorrowedState<'a>)) (index=0)
thread 'main' panicked at 'Box', librustc_errors/lib.rs:491:9
rust-lang/rust#50115

I really need this to work. What would be the recommended way to do this before this ICE issue gets fixed? :)

Btw, how long do you think it will take until this issue gets fixed?

@Boscop
Copy link

Boscop commented Apr 21, 2018

I found a workaround, pulling Dev outside of the <'a> quantification scope:

https://play.rust-lang.org/?gist=9b7907d94dfc28aabe12f5f90f8c4592&version=stable

pub trait DeviceSpecific {
	type Dev: Device;
}

pub trait DeviceApp<'a>: DeviceSpecific {
// 	type Dev: Device;

This compiles, but it's a combo of 2 ugly workarounds (I'd prefer not to split the trait like that) that wouldn't be necessary with proper (working) GATs, right?

Can you think of any solution that doesn't require me to split the trait like that? :)

@Boscop
Copy link

Boscop commented Apr 21, 2018

Ah no, it doesn't work with the impl of AppMgr :(

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	device: Option<T::Dev>,
	check_connection: AtFps,
	in_port_name: String,
	out_port_name: String,
	app: T,
}

impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {

	// error: cannot extract an associated type from a higher-ranked trait bound in this context

	pub fn process(&mut self, now: u64, v: &mut T::BorrowedState) -> Vec<T::Event> {
	//                                          ^^^^^^^^^^^^^^^^
		if self.check_connection.is_due(now) {
			let should_reconnect = if let Some(ref mut device) = self.device {
				!device.is_connected()
			} else {
				true
			};
			if should_reconnect {
				self.device = T::Dev::connect(&self.in_port_name, &self.out_port_name).ok();
			}
		}
		if let Some(ref mut device) = self.device {
			let app = &mut self.app;
			device.frame(now, |device, input| { app.process(device, input, v) })
		} else {
			vec![]
		}
	}
}

What I need is this:

// pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
pub struct AppMgr<T: DeviceApp> { // DeviceApp not depending on the lifetime 'a that process() gets called with
	// ...
}

// impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {
impl<T: DeviceApp> AppMgr<T> {
	// instantiating T::BorrowedState with <'a> from the scope that process() is called in
	pub fn process<'a>(&mut self, now: u64, v: &'a mut T::BorrowedState<'a>) -> Vec<T::Event> {
		// ...
	}
}

Is there any way to do this now? If not, will this work with proper working GATs?

@Boscop
Copy link

Boscop commented Apr 21, 2018

I found a way to impl AppMgr that works:

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	device: Option<T::Dev>,
	// ...
}

impl<T: for<'a> DeviceApp<'a>> AppMgr<T> {
	pub fn process<'a>(&mut self, now: u64, msgs: Vec<<T as DeviceApp<'a>>::Msg>, v: &'a mut <T as DeviceApp<'a>>::BorrowedState) -> Vec<<T as DeviceApp<'a>>::Event> {
		// ...

Here as reduced working example:
https://play.rust-lang.org/?gist=7522f287efe68c7b73efcddef1c5dd2a&version=stable

But I still wasn't happy about having to split the trait, so I thought, if I mandate that Dev: 'static, the compiler will be able to infer that for all 'a, <T as DeviceApp<'a>>::Dev will be the same, namely the same as <T as DeviceApp<'static>>::Dev (because Dev: 'static means it won't depend on 'a).
So I tried that, bringing Dev back into the DeviceApp trait:

/*
pub trait DeviceSpecific {
	type Dev: Device;
}

pub trait DeviceApp<'a>: DeviceSpecific {
*/
pub trait DeviceApp<'a> {
	type Dev: Device + 'static;

// ...

pub struct AppMgr<T: for<'a> DeviceApp<'a>> {
	// device: Option<T::Dev>,
	device: Option<<T as DeviceApp<'static>>::Dev>, // because Dev: 'static, i thought it will be unifyable with any 'a
	app: T,
	// ...
}

But now I get this error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
  --> src/main.rs:79:53
   |
79 |             device.frame(now, |device, input| { app.process(device, input, v) })
   |                                                     ^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime 'a as defined on the method body at 75:5...
  --> src/main.rs:75:5
   |
75 |     pub fn process<'a>(&mut self, now: u64, msgs: Vec<<T as DeviceApp<'a>>::Msg>, v: &'a mut <T as DeviceApp<'a>>::BorrowedState) -> Vec<<T as DeviceApp<'a>>::Event> {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: ...so that the expression is assignable:
           expected &mut <T as DeviceApp<'_>>::BorrowedState
              found &mut <T as DeviceApp<'a>>::BorrowedState
   = note: but, the lifetime must be valid for the static lifetime...
   = note: ...so that the expression is assignable:
           expected &mut <T as DeviceApp<'_>>::Dev
              found &mut <T as DeviceApp<'static>>::Dev

https://play.rust-lang.org/?gist=d4084c0a87890d4ed17d0491e453565b&version=stable

Why can't rustc unify <T as DeviceApp<'static>>::Dev> with <T as DeviceApp<'a>>::Dev> for all 'a, when it KNOWS that Dev: 'static (the type doesn't depend on 'a, so it can only be the same for all 'a)?


Actually, for some reason, in my real code I get a different (similar) error instead (with nightly 2018-03-06):

error: free region `'a` does not outlive free region `'static`
   --> src\devices\mod.rs:132:15
    |
132 |         if let Some(ref mut device) = self.device {
    |                     ^^^^^^^^^^^^^^

Any idea how to make it work with Dev inside the DeviceApp trait? :)

@burdges
Copy link

burdges commented Aug 6, 2018

I quite like the StreamingIterator example here, but another question: Should we permit higher kinded type parameters? As an example, you could borrow a data structure and use mut reborrows to iterate over it several times like:

fn process_elements<V: ?Sized,II>(x &mut V,f: fn<'a>(&mut V) -> II)
where
    II<'a>: Iterator<Item=&mut X>

or

fn process_elements<F,II>(f: &mut F)
where 
    for<'a> &'a mut F: Into<II<a'>>,
    for<'a> II<'a>: Iterator<Item=&'a mut X>

In principle, these could be done with impl Trait in certain positions, but this feature seems logically prior.

@dhardy
Copy link
Contributor

dhardy commented Dec 6, 2019

Re-link the tracking issue, which is not mentioned at either the top/bottom of this thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Tracker
Merge proposed
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.