🔬 Tracking issue for generic associated types (GAT)

[withoutboats]

This is a tracking issue for generic associated types (rust-lang/rfcs#1598)

TODO:

• Implement (mentoring instructions here)
• Document
• Stabilize

[nikomatsakis]

Here is a kind of implementation plan I will endeavor to keep updated.

• Step one: add support into the AST and pretty-printing
  • Probably the first step is to start parsing the new forms and to add support for them to the AST.
  • See this comment for more detailed thoughts here.
  • We should be able to write some parsing-only tests and also test the pretty-printer hir
  • When we get to HIR lowering, we can error out if any GAT are present
  • We can also do the feature gate then
• More to come

[nikomatsakis]

Let me start by writing about the AST in more detail. First let's discuss how it works today:

A type Foo: Bar [= Baz]; item in a trait definition is defined by this AST variant. That includes the bounds (Bar) and the (optional) default value Baz. The name is defined in the TraitItem struct.

A type Foo = Bar; item in a trait impl is defined by this AST variant -- that only includes the type Bar, because the Foo etc is defined in the ImplItem struct.

Methods are an interesting case because they can already be made generic. Those generic parameters are declared in the field Generics of the MethodSig structure. This is an instance of the Generics struct.

My take is that the best thing to do would be to "lift" Generics out from methods into the TraitItem (and ImplItem) so that it applies equally to all forms of trait and impl items. For now, we will not support generic constants, I guess, but honestly they probably just fall out from the work we are doing in any case, so they would be a small extension. I think the work will go better if we just plan for them now.

Perhaps a decent first PR would be to just do that change, keeping all other existing functionality the same. That is, we would more Generics into TraitItem (and ImplItem) and out of MethodSig. We would supply an empty Generics for non-methods. We would work through the existing code, plumbing the generics along as needed to make it work.

[sunjay]

@nikomatsakis Cool! Thank you so much! I started experimenting with this last night and I'm proud to say that I found the same places you pointed to in your comment about the AST. 😄 (Given that this was my first time in rustc, I count that as an accomplishment!)

I didn't think to lift generics into TraitItem. My approach was to put Generics into TraitItemKind::Type since that's where the type declaration is stored already. Your approach makes sense too so I'll work on implementing that. Since I am still completely new to this codebase, I'm interested to know what the pitfalls of my approach would be if it were used over the one you suggested. Could you give me some insight into your thought process? 😃

Here's the change I would have made:

pub enum TraitItemKind {
    // Generics aren't supported here yet
    Const(P<Ty>, Option<P<Expr>>),
    // `Generics` is already a field in `MethodSig`
    Method(MethodSig, Option<P<Block>>),
    // Added `Generics` here:
    Type(Generics, TyParamBounds, Option<P<Ty>>),
    Macro(Mac),
}

Edit: Answer by nikomatsakis on Gitter

regarding the pitfalls of putting them in type
I think that could work too
the reason that I was reluctant to do that
is that we really want to do the same things (in theory, at least) for methods and types
and -- as I said -- in principle I see no reason we couldn't do the same things for constants
I think if you just move the Generics to the type variant
that would probably work ok, but if you look about, right now we often have to do "one thing for types/consts, one thing for methods" precisely because they are different
so I suspect the code will just get more uniform
I'm not really sure how it will go to be honest =) -- it might be a pain
but a lot of times having things be more generic than they have to be isn't so bad, because you can insert span_bug! calls in the impossible cases for now (and later we come aoround and patch them up)

[nikomatsakis]

All right! Next step is to extend the parser. Here are a few tips. Let's start with trait items.

This routine parses trait items. We want to extend the case that handles associated types to also parse things like type Foo<....> = ...; (also maybe where-clauses). (The <...> is the "generics" that we just added to the AST.)

Currently it is using parse_ty_param, which basically parses something like T: Foo etc. We'll have to stop doing that, because the grammar for associated type declarations no longer matches that for type parameters. So we'll probably want to add something like parse_trait_item_assoc_ty. This can start as a kind of clone of parse_ty_param(), but then we'll want to modify it to invoke parse_generics() right about here. That routine will parse a generics declaration (<...>) if one is present, otherwise it just returns an empty generics. Then we want to add a call to parse where clauses right here -- you can model that on the call that occurs when parsing methods, note that the result is stored into the generics that we parsed earlier.

Once we've done that, we should be able to add some parsing tests. I would do that by making a directory like src/test/run-pass/rfc1598-generic-associated-types/ and adding files that you expect to successfully parse in there. Right now they won't work right, but that doesn't matter. Just add an empty main function. Then we can also add examples that should not parse into src/test/ui/rfc1598-generic-associated-types/ (see COMPILER_TESTS.md for directions on adding UI tests).

Something else -- we need to feature gate this work at this point, to avoid people using this stuff in stable builds. There are some instructions for adding a feature-gate here on forge (see the final section). We should add a visit_trait_item and visit_impl_item to the visitor in feature_gate.rs; if that item is not a method, but it has a non-empty generics, we can invoke gate_feature_post (example).

[nikomatsakis]

To setup name resolution, I think all we have to do is to get the proper "ribs" in place (the name resolution stuff organizes the sets of names that are in scope into ribs; each rib represents one binding level). e.g. for an impl:

impl<A,B> Foo<B> for Vec<A> {
   fn bar<T,U>(x: ...) { 
       for y in ... {
       }
   }
}

we would have the following ribs:

- <A,B> (from the impl)
   - <T,U> (from the `bar` method's generics)
      - `x` (from the parameter list)
          - `y` (from the let)

In general, modeling things on how methods work is not a bad idea. We might also do a bit of "future proofing" here, I suppose.

Here is the code that brings a method's type parameters into scope (this is for a method defined in a trait):

rust/src/librustc_resolve/lib.rs

Lines 1890 to 1892 in a35a3ab

	let type_parameters =
	HasTypeParameters(&trait_item.generics,
	MethodRibKind(!sig.decl.has_self()));

Whereas for a type defined in a trait, we are hardcoded to add an empty type parameter rib (NoTypeParameters):

rust/src/librustc_resolve/lib.rs

Lines 1897 to 1901 in a35a3ab

	TraitItemKind::Type(..) => {
	this.with_type_parameter_rib(NoTypeParameters, |this| {
	visit::walk_trait_item(this, trait_item)
	});
	}

Now that generics are in place on every trait/impl item, I think we probably want to remove the handling for type and extract the method handling so that it occurs at a higher level. For the items (e.g., const) where there are no generics, then the newly introduced rib ought to be empty and hence harmless (I hope).

Other points of interest:

• where impl item methods are setup
• where type in impls are setup

You get the idea.

@petrochenkov -- sound about right?

[petrochenkov]

@nikomatsakis

sound about right?

Everything looks correct.

[rickyhan]

Is anyone still working on this? It seems to me that this has a long way to go. GAT is my most desired RFC. If not, I'd love to contribute some tests...

[nikomatsakis]

@rickyhan so @Centril and @gavento were talking on the WG-traits gitter about divvying up the test work, but I don't know if any progress has been made. Maybe they can chime in. I think a PR would be welcome. =)

[quadrupleslap]

Sorry if I'm being stupid, but is this kind of thing going to be legal with GATs?

trait Sequencer {
    type Wrap<A>;
    fn chain<A, B, F>(Self::Wrap<A>, F) -> Self::Wrap<B>
        where F: FnOnce(A) -> Self::Wrap<B>;
    fn wrap<A>(A) -> Self::Wrap<A>;
}

[mark-i-m]

What's the status of this? I know that chalk recently gained gat support. Is that intended to land in rust soon?

[baloo]

@mark-i-m I gave it a shot last week. From my understanding, the syntax parser is there (although missing tests). But the "implementation" is not written yet. (see #44265 (comment) for more details)

[Boscop]

@quadrupleslap AIUI, that will be possible later, but at first, GATs will only support lifetime parameters..

[Centril]

@Boscop the RFC specifies that type parameters will also be supported.

[scalexm]

Does someone know the exact status of the implementation of the syntax in rustc? It seems mostly there, but higher ranked bounds generate ICEs:
http://play.rust-lang.org/?gist=a48959858ed5dd432c2396feae5c3cc1&version=nightly&mode=debug

I’d at least need the whole syntax to be implemented in order to advance on the chalkification work. If anybody is still working on the syntax, please let me know, or else I might go and fix it myself :)

[LukasKalbertodt]

To me it looks like communication is the major problem slowing down the implementation of GATs. There seem to be at least a few people interested in working on this, but no one really knows the exact implementation status and who is working on this already. What do you think about an Discord (or IRC?) channel just to talk about implementing GATs? I think that would certainly help.

Also, I could certainly contribute a few work days in the following weeks to work on this (but I don't really know anything about this part of the compiler yet).

[LukasKalbertodt]

So I asked for a dedicated channel on Discord. But instead of getting a channel, I learned a few things. I also searched for everything related to GATs. So I'll try to summarize all information regarding this feature; maybe it's useful to some people. But I don't know anything, so take this with a grain of salt! Also: please tell me if some information is incorrect so that I can fix it.

Summary of implementation efforts regarding GATs

• @sunjay landet two awesome PRs:

  • Generic Associated Types Parsing & Name Resolution (merged 2017-12-02)
  • Lifetime Resolution for Generic Associated Types (merged 2017-12-15)

• Then, for some time, not a lot happened. Niko then summarized what still needs to happen in this comment (2018-03-12). Basically (a) write tests, (b) fix bugs in/improve parser/front-end and (c) the implementation in the trait system. Point (a) was further explained in this comment (2018-03-13).

• @gavento landed Extend tests for RFC1598 (GAT) (merged 2018-05-11)

  • I'm not sure if Niko's comment about the tests already reflects changes made in this PR. The latest edit is from 2018-03-27, the same day @gavento opened their PR. (@gavento can you tell us about the status?)

Since then there weren't any UI tests added. And I can't find any other PRs directly related to GATs.

However, the point (c) from above (the trait system) is being worked on "in secret". As far as I understand it, the plan is to migrate to the new chalk-based trait solver soon and not make GATs work on the old system. Integration of the new trait solver is tracking by the "Chalkification" tracking issue. There have been quite a few PRs related to chalk and chalkification. Notably, there is a chalk PR called "Finish implementing GATs" (merged 2018-05-24). So it seems like the core system for GATs is already in place.

That said, the "GATs" in chalk are a prototype implementation and using it in rustc is not just a use chalk;. As @scalexm told me: "There seems to be quite a lot [to do]".

---

For more information and to help out, it's probably useful to take a look into the #wg-traits Discord channel and the traits WG tracking issue.

[LukasKalbertodt]

So @nikomatsakis just created the #wg-traits-gat channel on the rust-lang discord server (join here). It would be great if we could get everyone who wants to help in there. Plus a few people who know about the status of this feature (in particular, what still needs to be done and where we could help). That should make communication easier and faster, especially for people like me who are not yet deeply involved/part of the traits WG :)

[alexreg]

Is there any update on this recently? Are we waiting for Chalk integration into the compiler? (Perhaps there's a separate issue for that.)

[shepmaster]

Perhaps there's a separate issue for that.

#48049