Stabilize return type notation (RFC 3654)

[compiler-errors]

Return Type Notation (RTN) Stabilization Report

Stabilization summary

This PR stabilizes return-type notation in where clause and item bound position, both in path and associated type bound forms for methods in traits that have lifetime generics.

fn foo<T, U>()
where
    // Associated type bound
    T: Trait<method(..): Send + 'static>,
    // Path bound
    U: Trait,
    U::method(..): Send + 'static,
{}

trait Trait {
    // In GAT bounds.
    type Item: Trait<method(..): Send + 'static>;
}

// In opaque item bounds too.
fn rpit() -> impl Foo<method(..): Send + 'static>;

This gives users a way to add trait bounds to the anonymous generic associated types (GATs) that are generated from the return-position impl trait in trait (RPITIT) desugaring.

Motivation

Rust now supports AFITs and RPITITs, but we currently lack the ability for users to declare bounds on the anonymous types returned by such functions at the usage site. This is often referred to as the Send bound problem, since it commonly affects futures which may only conditionally be Send depending on the executor that they are involved with.

Return type notation gives users an intuitive way to solve this problem. See the alternatives section of the RFC for other proposed solutions.

This feature makes AFIT more useful in APIs, where users can now bound their methods appropriately at the usage site rather than having to provide a constrained definition like -> impl Future<Output = T> + Send within the trait.

Major design decisions since the RFC

There were no major changes to the design, but there was some consideration whether the bound should be spelled with (..) or (). The implementation commits to (..) since that saves space for explicitly specified parameters in the future like T::method(i32): Trait which can be used to only bound subsets of the RTN type.

What is stabilized?

As mentioned in the summary, RTN is allowed as the self type of a where clause anywhere that a where clause is allowed. This includes shorthand syntax, like where T::method(..): Send, where the trait is not specified, in the same situations that it would be allowed for associated types.

RTN is also allowed in associated type bound position, and is allowed anywhere an associated type bound is allowed, except for dyn Trait types.

Linting

This PR also removes the async_fn_in_trait lint. This lint was introduced to discourage the usage of async fn in trait since users were not able to add where clause bounds on the futures that these methods returned.

This is now possible for functions that have only lifetime generics. The only caveat here is that it's still not possible for functions with type or const generics. However, I think that we should accept that corner case, since the intention was to discourage the usage before RTN was introduced, and not to outright ban AFIT usage. See #116184 for some context.

I could change my mind on this part, but it would still be quite annoying to have to keep this lint around until we fix RTN support for methods with type or const generics.

What isn't stabilized? (a.k.a. potential future work)

RTN is not allowed as a free-standing type. This means that it cannot be used in ADTs (structs/enums/unions), parameters, let statements, or turbofishes.

RTN can only be used when the RPITIT (return-position impl trait in trait) is the outermost type of the return type, i.e. -> impl Sized but not -> Vec<impl Sized>. It may also be used for AFIT (async fn in trait), since that desugars to -> impl Future<Output = T>.

RTN is only allowed for methods with only lifetimes. Since the RTN syntax expands to a for<...> binder of all of the generics of the associated function, and we do not have robust support for non-lifetime binders yet, we reject the case where we would be generating a for<T> binder for now. This restriction may be lifted in the future.

RTN cannot be used to bound the output futures from calling AsyncFn* traits yet. This requires some design work. See: https://rust-lang.github.io/rfcs/3668-async-closures.html#interaction-with-return-type-notation-naming-the-future-returned-by-calling

Implementation summary

This feature is generally a straightforward extension of associated types and associated type bounds, but adjusted (in the presence of (..)-style generics) to do associated item resolution in the value namespace to look up an associated function instead of a type.

In the resolver, if we detect a path that ends in (..), we will attempt to resolve the path in the value namespace:

rust/compiler/rustc_resolve/src/late.rs

Lines 800 to 810 in 30f168e

	// If we have a path that ends with `(..)`, then it must be
	// return type notation. Resolve that path in the *value*
	// namespace.
	let source = if let Some(seg) = path.segments.last()
	&& let Some(args) = &seg.args
	&& matches!(**args, GenericArgs::ParenthesizedElided(..))
	{
	PathSource::ReturnTypeNotation
	} else {
	PathSource::Type
	};

In "resolve bound vars", which is the part of the compiler which is responsible for resolving lifetimes in HIR, we will extend the (explicit or implicit) for<> binder of the clause with any lifetimes params from the method:

rust/compiler/rustc_hir_analysis/src/collect/resolve_bound_vars.rs

Lines 2002 to 2019 in 30f168e

	// When we have a return type notation type in a where clause, like
	// `where <T as Trait>::method(..): Send`, we need to introduce new bound
	// vars to the existing where clause's binder, to represent the lifetimes
	// elided by the return-type-notation syntax.
	//
	// For example, given
	// ```
	// trait Foo {
	// async fn x<'r>();
	// }
	// ```
	// and a bound that looks like:
	// `for<'a, 'b> <T as Trait<'a>>::x(): Other<'b>`
	// this is going to expand to something like:
	// `for<'a, 'b, 'r> <T as Trait<'a>>::x::<'r, T>::{opaque#0}: Other<'b>`.
	//
	// We handle this similarly for associated-type-bound style return-type-notation
	// in `visit_segment_args`.

We similarly handle RTN bounds in associated type bounds:

rust/compiler/rustc_hir_analysis/src/collect/resolve_bound_vars.rs

Lines 1792 to 1804 in 30f168e

	// If the args are parenthesized, then this must be `feature(return_type_notation)`.
	// In that case, introduce a binder over all of the function's early and late bound vars.
	//
	// For example, given
	// ```
	// trait Foo {
	// async fn x<'r, T>();
	// }
	// ```
	// and a bound that looks like:
	// `for<'a> T::Trait<'a, x(..): for<'b> Other<'b>>`
	// this is going to expand to something like:
	// `for<'a> for<'r> <T as Trait<'a>>::x::<'r, T>::{opaque#0}: for<'b> Other<'b>`.

In HIR lowering, we intercept the type being lowered if it's in a where clause:

rust/compiler/rustc_hir_analysis/src/hir_ty_lowering/bounds.rs

Lines 434 to 436 in 30f168e

	/// Lower a type, possibly specially handling the type if it's a return type notation
	/// which we otherwise deny in other positions.
	pub fn lower_ty_maybe_return_type_notation(&self, hir_ty: &hir::Ty<'tcx>) -> Ty<'tcx> {

.

Otherwise, we unconditionally reject the RTN type, mentioning that it's not allowed in arbitrary type position. This can be extended later to handle the lowering behavior of other positions, such as structs and unsafe binders.

The rest of the compiler is already well equipped to deal with RPITITs, so not very much else needed to be changed.

Tests

"Positive tests":

• "basic" test: tests/ui/associated-type-bounds/return-type-notation/basic.rs.
• Resolution conflicts prefer methods with .. syntax: tests/ui/associated-type-bounds/return-type-notation/namespace-conflict.rs.
• Supertraits: tests/ui/async-await/return-type-notation/supertrait-bound.rs.
• "Path"-like syntax: tests/ui/associated-type-bounds/return-type-notation/path-works.rs.
• RTN where clauses promoted into item bounds: tests/ui/associated-type-bounds/implied-from-self-where-clause.rs.
• Exercising higher ranked bounds: tests/ui/associated-type-bounds/return-type-notation/higher-ranked-bound-works.rs.

"Negative tests":

• Bad inputs or return types: tests/ui/associated-type-bounds/return-type-notation/bad-inputs-and-output.rs.
• Rejected in expression position: tests/ui/associated-type-bounds/return-type-notation/bare-path.rs.
• Reject type equality: tests/ui/associated-type-bounds/return-type-notation/equality.rs.
• Rejected on non-RPITITs: tests/ui/associated-type-bounds/return-type-notation/non-rpitit.rs + tests/ui/associated-type-bounds/return-type-notation/not-a-method.rs.
• Associated type bound ambiguity: tests/ui/async-await/return-type-notation/super-method-bound-ambig.rs.
• Shorthand syntax ambiguity: tests/ui/associated-type-bounds/return-type-notation/path-ambiguous.rs.
• Test rejecting types and consts: tests/ui/async-await/return-type-notation/ty-or-ct-params.rs.

Remaining bugs, FIXMEs, and open issues

• Unable to write blanket impl for subtrait on RPITIT trait #124729 - Is just a manifestation of a bug with associated types.
• Mysterious "higher-ranked lifetime error" with async fn in trait and return-type notation #110963 - Is just a manifestation of general async block + auto trait bug.
• hrtb + infer types break auto traits with return type notation  #109924 - Same as above.
• rustdoc: support RTN once its syntax is somewhat finalized  #123996 - Rustdoc support discussed below.

What other unstable features may be exposed by this feature?

This feature gives users a way to name the RPIT of a method defined in an impl, but only in where clauses.

Tooling support

Rustfmt: ✅ Supports formatting (..)-style generics.

Clippy: ✅ No support needed, unless specific clippy lints are impl'd to care for RTN specifically.

Rustdoc: ❓ #137956 should fix support for RTN locally. Cross-crate re-exports are still broken, but that part of rustdoc (afaict) needs to be overhauled in general.

Rust-analyzer: ❓ There is parser support. I filed an issue for improving support in the IDE in rust-lang/rust-analyzer#19303.

History

RTN was first implemented in associated type bounds in #109010.

It was then implemented in where T::method(..): Send style bounds in #129629.

The RFC was proposed in rust-lang/rfcs#3654.

There was a call for testing here: https://blog.rust-lang.org/inside-rust/2024/09/26/rtn-call-for-testing.html.

History

• Initial support for return type notation (RTN) #109010
• Print the path of a return-position impl trait in trait when return_type_notation is enabled #115624
• Resolve associated item bindings by namespace #118668
• AstConv: Refactor lowering of associated item bindings a bit #121221
• Put the dots back in RTN pretty printing #127783
• Implement Return Type Notation (RTN)'s path form in where clauses #129629
• No longer mark RTN as incomplete #130705
• Robustify and genericize return-type-notation resolution in resolve_bound_vars #132047

Acknowledgments

Thanks to @nikomatsakis for writing the RFC for RTN, and to various reviewers for reviewing the various PRs implementing parts of this feature.

Pending items before stabilization

• Land rustdoc support
• Land or defer rust-analyzer support
• Finalize style for RTN
• Reference work

[rustbot]

r? @wesleywiser

rustbot has assigned @wesleywiser.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[compiler-errors]

cc @rust-lang/lang and @rust-lang/types (though this PR doesn't do that much interesting on the types side other than giving us a new way of naming RPITITs).

[compiler-errors]

cc @rust-lang/rust-analyzer

[theemathas]

Of note, the return type notation for specifying the Future in the AsyncFn traits doesn't seem to have been implemented yet. https://rust-lang.github.io/rfcs/3668-async-closures.html#interaction-with-return-type-notation-naming-the-future-returned-by-calling

[compiler-errors]

Yeah that will need to get done later as its semantics are quite different. I'll put it in the report.

[JohnDowson]

Man, that syntax is not great. Really gives ammo to people who are like "rust is a pile of confusing sigils". Which isn't really a good reason to reject something, but I'd rather we had a more generally applicable and less confusing syntax for decltype-like type referencing.

Particularly given relative rarity of cases where one would see or use this syntax, I think using something that looks like a valid UFCS call expression will be a perpetually renewing source of confusion amongst the community.

It's also rather weird, given the fact that Fn traits more or less already have an assoc type for the output.
where <T::method as Fn<...>>::Output: Frobnifier would be an infinitely more consistent way to express this!

[oli-obk]

Please see https://rust-lang.github.io/rfcs/3654-return-type-notation.html#what-other-syntax-options-were-considered for the different syntaxes that have been considered (among it the one you propose), and why they have been rejected. There was quite some discussion about these

• RFC: Return Type Notation rfcs#3654
• https://hackmd.io/@rust-lang-team/rypohY76a (lang team meeting on RTN syntax)
• https://hackmd.io/@rust-lang-team/ByUojGAn6 (lang team doc on syntax options)
• https://hackmd.io/@rust-lang-team/BkuJDYmpp (follow-up to lang team doc on syntax options)
• https://hackmd.io/@rust-lang-team/BJlMG7ps6 (meeting specifically about concerns about the syntax)
• https://hackmd.io/@rust-lang-team/rkbHX_-cT (original lang team design meeting)

TLDR: the alternatives have other problems of their own, and this syntax is at least concise.

[JohnDowson]

Please see https://rust-lang.github.io/rfcs/3654-return-type-notation.html#what-other-syntax-options-were-considered for the different syntaxes that have been considered (among it the one you propose), and why they have been rejected.

It doesn't actually list the one I proporse, only the non-FQ item::Output. Perhaps it is mentioned somewhere in the meeting notes.

Edit:
After skimming through all of the linked notes, I feel even stronger that the "strangeness budget" for the syntax was underconsidered. It's a very weird syntax, that has all of the same downsides as assoc projection (and really is just a way to spell assoc projection specifically for Fn instances), but replaces "a bit wordy" with "inconsistent with the rest of the language and, makes Fn* traits even more ✨magical✨ than they already are".

[JohnDowson]

Leaving syntax alone, I also want to argue that RTN is solving the wrong issue.

We already have a way to refer to associated types, what's lacking, is the ability to refer to a function's type, or, more generally, to a type of a non-type item.

[compiler-errors]

A lot of the arguments I laid out in #109417 (comment) apply to something like <T::method as Fn<...>>::Output, specifically:

1. One minor nitpick is that <T as Fn<()>>::Output isn't the right way to spell the bound, since Output is an associated type of FnOnce, unless we were to introduce a new hack to UFCS here.
2. Resolution inconsistency with type vs. value namespaces in the resolver, which would need to be hacked specifically for FnOnce's self type to resolve in the value namespace.
3. Associated types need all of their generics specified. If we were to add some sort of elision like <T as FnOnce<..>>::Output where .. is syntax that means "figure out what I meant here", it would mean complicating lowering associated types further.
4. The tmplicit higher-rankedness of RTN types means we'd need to hack for the FnOnce associated type lowering in where-clause position to extend the where-clause's for<> binder with the associated lifetimes of the method.

RTN syntax means that these four problems either don't apply or syntactically distinguishes them from the compiler's perspective enough that handling them specially is very easy. I think also conceptually, having a distinct syntax for distinct behavior is better for the user, too.

[bors]

☔ The latest upstream changes (presumably #138464) made this pull request unmergeable. Please resolve the merge conflicts.

[traviscross]

In testing, I notice that in the case of ambiguity between an item of a subtrait and a supertrait, RTN will prefer the method from the subtrait. E.g.:

Playground link

This is consistent with how associated type bounds and equality constraints work in general, so it's what I'm sure we would expect. E.g.:

Playground link

It's just interesting to note, given our direction with supertrait item shadowing (tracked in #89151). I don't immediately see a test for it, and it may be worth that and worth us mentioning it somewhere. It doesn't seem that the RFC specified this one way or the other.

cc @Amanieu

[traviscross]

Having reviewed all the tests and beat on this myself, this all looks in order to me. In terms of tooling, it looks like the rustdoc support has now been merged. Rust-analyzer has parser support, which is usually enough for us to ship a feature, and there's a PR up with further support. We'll still need an approved Reference PR before we merge this stabilization, but we can go ahead and get the ball rolling.

Based on the fact that CE, who would know, tagged this only for lang and then mentioned, "this PR doesn't do that much interesting on the types side other than giving us a new way of naming RPITITs", I'm going to propose a lang-only FCP here.

@rfcbot fcp merge

Thanks to @compiler-errors for pushing forward this important work.

[nikomatsakis]

@rfcbot reviewed

I've been playing with this. I am satisfied with the behavior. I still would like to do more QA but I'm checking my box on the expectation that this will not find anything very surprising. =)

[joshtriplett]

This is awesome and I'm thrilled for us to ship it.

@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[tmandry]

RTN cannot be used to bound the output futures from calling AsyncFn* traits yet. This requires some design work.

Is the design work that the syntax is uncertain? I assume something like this would be plausible..

fn foo<F>(f: F)
where
    F: AsyncFn() -> i32,
    F(..): Send,
{}

I understand from the RFC that it's a bit nuanced in that it desugars to two bounds, but I think that's fine. Anyway, just want to make sure there's a plausible syntax that we don't have major known issues for as part of due diligence before we ship this.

[tmandry]

Since I'm expecting a positive response to my above comment, I'll do

@rfcbot reviewed

I'm very grateful to @compiler-errors for all the meticulous work you've done to make this feature a reality, and thanks @nikomatsakis for the RFC!

[compiler-errors]

I understand from the RFC that it's a bit nuanced in that it desugars to two bounds, but I think that's fine. Anyway, just want to make sure there's a plausible syntax that we don't have major known issues for as part of due diligence before we ship this.

Yes.

Well, first of all one may argue that T(..): Send AsyncFn* bounds are just a totally different feature than RTN, other than sharing the (..) syntax.

And as for the T(..): Send syntax itself, there's some implementation nuances around the way it interacts with the bound lookup and how it interacts with higher-ranked bounds, but I don't think there's any major issues that would affect the "normal" RTN we're stabilizing today.

[lcnr]

RTN allows naming opaques in new places in the type system, e.g. in coherence checking

#![feature(return_type_notation)]
trait Trait {
    async fn foo() {}
}
impl Trait for u32 {}

// this overlaps as expected, so nothing interesting is happening in this example.
trait Other {}
impl<T> Other for T
where
    T: Trait,
    T::foo(..): Send,
{}
impl Other for u32 {}

This very much needs a T-Types FCP and I am not confident in how this interacts with the remaining blockers for TAIT itself.

[lcnr]

A non-trivial example which I believe only fails due an unrelated-ish cycle error, please add as a test

#![feature(return_type_notation)]
trait Trait {
    fn foo() -> impl Copy
    where
        Self::foo(..): Copy
    {
        String::from("temporary")
    }
}

impl Trait for () {}

fn main() {
    let ret = <()>::foo();
    drop((ret, ret))
}

[lcnr]

RTN can leak to be used pretty everywhere by adding some boilerplate code.

#![feature(return_type_notation)]
trait Trait {
    fn foo() -> impl Sized;
}

trait Id {
    type This: ?Sized;
}
impl<T: ?Sized> Id for T {
    type This = T;
}

trait GetOpaque {
    type Opaque: ?Sized;
}
impl<T: ?Sized, U> GetOpaque for T
where 
    T: Trait,
    T::foo(..): Id<This = U>,
{
    type Opaque = U;
}
type FooTypePos<T> = <T as GetOpaque>::Opaque;

impl Trait for u8 {
    fn foo() -> impl Sized {
        || ()
    }
}
struct ContainsOpaque(FooTypePos<u8>);

[tmandry]

@rfcbot concern needs-types-fcp

EDIT: @compiler-errors restarted FCP and I re-checked all the lang team boxes from the previous proposal.

This very much needs a T-Types FCP and I am not confident in how this interacts with the remaining blockers for TAIT itself.

From the lang side the major blocker I recall was having to worry about which uses are defining or not (we wanted to mark these with the #[defines] attribute, which hit implementation snags). This doesn't have that problem, because there is only one defining use and it's obvious what it is. cc @traviscross and @oli-obk who were driving this feature and might know of other blockers.

I'm not aware of the blockers on the T-types side.

[compiler-errors]

Well instead of filing a concern, let's just cancel this FCP and re-FCP it as lang + types.

[compiler-errors]

@rfcbot fcp cancel

[rfcbot]

@compiler-errors proposal cancelled.

[compiler-errors]

@rfcbot fcp merge

[rfcbot]

Team member @compiler-errors has proposed to merge this. The next step is review by the rest of the tagged team members:

• @BoxyUwU
• @compiler-errors
• @jackh726
• @joshtriplett
• @lcnr
• @nikomatsakis
• @oli-obk
• @scottmcm
• @spastorino
• @tmandry
• @traviscross

Concerns:

• accidentally-similar-to-type-position-rtn (Stabilize return type notation (RFC 3654) #138424 (comment))
• accidentally stabilizing TAIT/ATPIT (Stabilize return type notation (RFC 3654) #138424 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[lcnr]

Formalizing the above concern

@rfcbot concern accidentally stabilizing TAIT/ATPIT

RTN allows naming opaque types in arbitrary positions. This means we get nearly all implementation issues preventing TAIT, at least in theory. We either have to wait for them to get fixed (which I am working on right now), or somehow make sure users (ab)using RTN won't cause issues as we're working towards the stabilization of TAIT/the next-generation trait solver.

[traviscross]

Great catch. Extremely interesting. In retrospect, "of course that would be possible."

I'm going to file an overlapping lang concern,

@rfcbot concern accidentally-similar-to-type-position-rtn

though it amounts to the same thing, since clearly we didn't mean to do this and so we should discuss. Procedurally, though, I'm expecting we'll just end up canceling the proposal and later making a new proposed FCP if the answer to this is likely to be substantial (or delayed) enough that we'd want people to check back off anyway.

[compiler-errors]

accidentally-similar-to-type-position-rtn

@traviscross: What do you even mean by this concern? I'm not totally certain why you're drawing a parallel to type position RTN or suggesting that that's a concern, since AFAICT the complications of that feature are things like "what do we do with the higher-ranked lifetimes of the type", which isn't AFAICT achievable with the example that lcnr shared, since it uses an associated type bound which necessarily means that it's not a higher-ranked type (e.g.).

When @lcnr filed the concern "accidentally stabilizing TAIT/ATPIT", my understanding is that it's primarily from an implementation perspective that this makes it easier to do certain things that will possibly give us headaches in the new trait solver (some of which are possible today, but a bit more difficult to do b/c they involve recursive calls since that's the only way to get a value of an RPIT type on stable, and some of which are not possible, like putting RTN types into where clauses).

So I'm not sure why this immediately warrants a parallel lang concern being filed unless you can explain what that concern means and why it's different? While I'm sympathetic to blocking this at least¹ on building some confidence w.r.t. making sure we're not giving ourselves a future headache w/ the new trait solver and opaques, I really still can't see what this has to do with the lang side of the feature.

Footnotes

1. which is a pretty wide gamut: it's everything from "ok, this is actually probably not a concern in practice" all the way to "we need to wait until we've totally fixed TAIT in the new solver". ↩

[traviscross]

Please don't read too much into it. It's just a placeholder for being sure that we discuss this new information on the lang side and, in resolving it, signal that we're willing to proceed anyway.