Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure async trait impls are async (or otherwise return an opaque type) #104592

Merged
merged 1 commit into from
Dec 15, 2022

Conversation

ComputerDruid
Copy link
Contributor

@ComputerDruid ComputerDruid commented Nov 19, 2022

As a workaround for the full #[refine] semantics not being implemented
yet, forbit returning a concrete future type like Box<dyn Future> or a
manually implemented Future.

-> impl Future is still permitted; while that can also cause
accidental refinement, that's behind a different feature gate
(return_position_impl_trait_in_trait) and that problem exists
regardless of whether the trait method is async, so will have to be
solved more generally.

Fixes #102745

@rustbot
Copy link
Collaborator

rustbot commented Nov 19, 2022

r? @fee1-dead

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 19, 2022
@ComputerDruid
Copy link
Contributor Author

r? @compiler-errors

cc @tmandry

While this would help unblock stabilization we might prefer not to land this if we're worried about artificially blocking testing.

@rustbot rustbot assigned compiler-errors and unassigned fee1-dead Nov 19, 2022
Comment on lines 356 to 357
if tcx.asyncness(trait_m.def_id) == hir::IsAsync::Async {
if tcx.asyncness(impl_m.def_id) != hir::IsAsync::Async {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not make it both ways?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's not really a problem with using an async impl if the trait is -> impl Future + '_. I think it's not possible to create a refinement hazard with an async fn, but even if it is, that's already a problem with RPITIT that'll have to be solved before it can be stabilized, and I expect the solution to that to automatically cover whatever hypothetical refinement problem this might expose.

And we don't have to get it right in order to stabilize async_fn_in_trait since the RPITIT stuff is behind a different feature gate.

@compiler-errors
Copy link
Member

While this would help unblock stabilization we might prefer not to land this if we're worried about artificially blocking testing.

Can you elaborate what you mean here?

@rust-log-analyzer

This comment has been minimized.

@compiler-errors
Copy link
Member

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 19, 2022
@tmandry
Copy link
Member

tmandry commented Nov 19, 2022

Hm, I forgot about this. I think I'd prefer to allow the actual mixing and matching allowed in the RFC, even if it means we have to allow too many other things in the meantime. This made sense as an early implementation step, but now that the call for testing blog post is out (and even uses rpitit) I think it will really restrict the cases where it makes sense to use async in a trait.

How difficult would it be to make the check allow mixing async fn() with fn() -> impl Future + '_?

@compiler-errors
Copy link
Member

How difficult would it be to make the check allow mixing

@tmandry, Can you explain what kind of mixing you mean? Right now, the RPITIT feature allows both impl Future trait + async fn impl and async fn trait + impl Future impl. If we want both, then nothing needs to be done, and this can just be closed.

@tmandry
Copy link
Member

tmandry commented Nov 19, 2022

Sorry, I said that this was allowed in the async fn RFC, but it's only mentioned as a future possibility (that requires return position impl Trait, which is not RFC'd yet.)

Ideally I would like to allow mixing both ways, so that implementers of an async fn in a trait can run code before returning the future. But the impl Future signature should match the async fn desugaring exactly, which means the future should capture all input lifetimes. (This could also be made more clear in the RFC.) For example,

trait Foo {
    async fn foo(&self, x: &u32);
}

impl<'blah> Foo for &'blah () {
    // Accepted: Matches desugared form (these are equivalent)
    fn foo<'me, 'x>(&'me self, x: &'x u32) -> impl Future<Output = ()> + 'me + 'x { async {} }
    fn foo<'x>(&'me self, x: &'x u32) -> impl Future<Output = ()> + '_ + 'x { async {} }

    // Not accepted: return type must include all input lifetimes
    fn foo<'x>(&self, x: &'x u32) -> impl Future<Output = ()> + '_ { async {} }
    fn foo<'x>(&self, x: &'x u32) -> impl Future<Output = ()> + 'x { async {} }
    fn foo<'x>(&self, x: &'x u32) -> impl Future<Output = ()> { async {} }

    // Not accepted: return type must not include additional lifetimes, like `'static`
    fn foo<'x>(&self, x: &'x u32) -> impl Future<Output = ()> + '_ + 'x + 'static { async {} }

    // Not accepted (this already works): return type must specify `Output`
    fn foo<'x>(&self, x: &'x u32) -> impl Future + '_ + 'x { async {} }

    // This one is acceptable because the extra `+ 'blah` bound is already implied by `+ '_`
    // (anything that outlives a reference to `self` must also outlive `self`),
    // but it's also okay to give an error here for now.
    fn foo<'x>(&self, x: &'x u32) -> impl Future<Output = ()> + '_ + 'x + 'blah { async {} }
}

The rationale for this is that according to rust-lang/rfcs#3245, any other form in the impl should require #[refine]. Until this RFC is implemented we should disallow the divergence altogether.

The same would apply for the other direction; this implies that you must use the correct desugared form in the trait to be able to use async fn in any impl. This is already enforced. The async fn desugaring is maximally conservative in that it includes all input lifetimes in the output type, so there is no way to satisfy a different signature using async fn. Right now it looks like this direction is not actually accepted even if you do write the desugaring correctly:

trait Foo {
    fn foo<'me, 'x>(&'me self, x: &'x u32) -> impl Future<Output = ()> + 'me + 'x;
}

impl Foo for () {
    async fn foo(&self, x: &u32) {}
    // ^ error[E0495]: cannot infer an appropriate lifetime for lifetime parameter '_ in generic type due to conflicting requirements
}

...but it's also less important (you can almost always just switch to using async fn in the trait; this is only a problem if you want to do some magic in a default implementation).

@tmandry
Copy link
Member

tmandry commented Nov 19, 2022

I should add there is one kind of divergence we should allow, auto traits. This actually works today, and must work before stabilizing return_position_impl_trait_in_trait:

trait Foo {
    fn foo(&self) -> impl Future<Output = ()> + '_ + Send;
}

impl Foo for () {
    async fn foo(&self) { }
}

I suppose it's okay if we have to break it in the interim, though. (I did mention that this would work in the blog post, but it's not critical, I can take that out!)

@tmandry
Copy link
Member

tmandry commented Nov 19, 2022

After typing this out I realize this amounts to a lot of work :). Looking at the PR again, I think it's find to land except we should just accept any -> impl Future, since that's behind a different feature flag. (Returning e.g. Pin<Box<dyn Future>> should not be accepted, as is already the case in this PR.)

That would be enough to ensure that the behavior is correct when only async_fn_in_trait is enabled, without removing flexibility from those who want to experiment with the (further off) return_position_impl_trait_in_trait.

@rustbot rustbot added the A-translation Area: Translation infrastructure, and migrating existing diagnostics to SessionDiagnostic label Dec 6, 2022
@rustbot
Copy link
Collaborator

rustbot commented Dec 6, 2022

rustc_error_messages was changed

cc @davidtwco, @compiler-errors, @JohnTitor, @estebank, @TaKO8Ki

@ComputerDruid ComputerDruid changed the title Ensure async trait impls also use async Ensure async trait impls are async (or otherwise return an opaque type) Dec 6, 2022
@rust-log-analyzer

This comment has been minimized.

@compiler-errors
Copy link
Member

@bors r+

approving for now, we can always tweak or roll back semantics if needed.

@bors
Copy link
Contributor

bors commented Dec 14, 2022

📌 Commit 259c37e has been approved by compiler-errors

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 14, 2022
@tmandry
Copy link
Member

tmandry commented Dec 14, 2022

@bors rollup

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 14, 2022
…ler-errors

Ensure async trait impls are async (or otherwise return an opaque type)

As a workaround for the full `#[refine]` semantics not being implemented
yet, forbit returning a concrete future type like `Box<dyn Future>` or a
manually implemented Future.

`-> impl Future` is still permitted; while that can also cause
accidental refinement, that's behind a different feature gate
(`return_position_impl_trait_in_trait`) and that problem exists
regardless of whether the trait method is async, so will have to be
solved more generally.

Fixes rust-lang#102745
@matthiaskrgr
Copy link
Member

@bors r- failed in a rollup :(
may just need a rebase / compilier errors fixed

https://github.com/rust-lang/rust/actions/runs/3699320469/jobs/6266555110#step:26:1115


   Compiling rustc_const_eval v0.0.0 (/checkout/compiler/rustc_const_eval)
   Compiling rustc_lint v0.0.0 (/checkout/compiler/rustc_lint)
   Compiling rustc_traits v0.0.0 (/checkout/compiler/rustc_traits)
   Compiling rustc_ty_utils v0.0.0 (/checkout/compiler/rustc_ty_utils)
   Compiling rustc_hir_analysis v0.0.0 (/checkout/compiler/rustc_hir_analysis)
error[E0532]: expected tuple struct or tuple variant, found unit variant `ty::Opaque`
   --> compiler/rustc_hir_analysis/src/check/compare_method.rs:339:13
    |
339 |             ty::Opaque(..) => {
    |             ^^^^^^^^^^^^^^
    |
   ::: /checkout/compiler/rustc_type_ir/src/sty.rs:42:5
    |
42  |     Opaque,
    |     ------ `ty::Opaque` defined here
    |
help: use this syntax instead
    |
339 |             ty::Opaque => {
    |             ~~~~~~~~~~
help: consider importing one of these items instead
    |
1   | use crate::check::compare_method::infer::outlives::components::Component::Opaque;
    |
1   | use crate::check::compare_method::infer::region_constraints::GenericKind::Opaque;
    |
1   | use rustc_infer::infer::outlives::components::Component::Opaque;
    |
1   | use rustc_infer::infer::region_constraints::GenericKind::Opaque;
    |
      and 2 other candidates
help: if you import `Opaque`, refer to it directly
    |
339 -             ty::Opaque(..) => {
339 +             Opaque(..) => {
    |

For more information about this erro

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 14, 2022
@compiler-errors
Copy link
Member

Ah yeah, ty::Opaque(..) needs to be turned into ty::Alias(ty::Opaque, ..)

As a workaround for the full `#[refine]` semantics not being implemented
yet, forbit returning a concrete future type like `Box<dyn Future>` or a
manually implemented Future.

`-> impl Future` is still permitted; while that can also cause
accidental refinement, that's behind a different feature gate
(`return_position_impl_trait_in_trait`) and that problem exists
regardless of whether the trait method is async, so will have to be
solved more generally.

Fixes rust-lang#102745
@compiler-errors
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Dec 15, 2022

📌 Commit da98ef9 has been approved by compiler-errors

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 15, 2022
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 15, 2022
…ler-errors

Ensure async trait impls are async (or otherwise return an opaque type)

As a workaround for the full `#[refine]` semantics not being implemented
yet, forbit returning a concrete future type like `Box<dyn Future>` or a
manually implemented Future.

`-> impl Future` is still permitted; while that can also cause
accidental refinement, that's behind a different feature gate
(`return_position_impl_trait_in_trait`) and that problem exists
regardless of whether the trait method is async, so will have to be
solved more generally.

Fixes rust-lang#102745
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 15, 2022
…ler-errors

Ensure async trait impls are async (or otherwise return an opaque type)

As a workaround for the full `#[refine]` semantics not being implemented
yet, forbit returning a concrete future type like `Box<dyn Future>` or a
manually implemented Future.

`-> impl Future` is still permitted; while that can also cause
accidental refinement, that's behind a different feature gate
(`return_position_impl_trait_in_trait`) and that problem exists
regardless of whether the trait method is async, so will have to be
solved more generally.

Fixes rust-lang#102745
bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 15, 2022
…iaskrgr

Rollup of 11 pull requests

Successful merges:

 - rust-lang#104592 (Ensure async trait impls are async (or otherwise return an opaque type))
 - rust-lang#105623 (Fix `-Z print-type-sizes` for generators with discriminant field ordered first)
 - rust-lang#105627 (Auto traits in `dyn Trait + Auto` are suggestable)
 - rust-lang#105633 (Make `report_projection_error` more `Term` agnostic)
 - rust-lang#105683 (Various cleanups to dest prop)
 - rust-lang#105692 (Add regression test for rust-lang#104678)
 - rust-lang#105707 (rustdoc: remove unnecessary CSS `kbd { cursor: default }`)
 - rust-lang#105715 (Do not mention long types in E0599 label)
 - rust-lang#105722 (more clippy::complexity fixes)
 - rust-lang#105724 (rustdoc: remove no-op CSS `.scrape-example .src-line-numbers { margin: 0 }`)
 - rust-lang#105730 (rustdoc: remove no-op CSS `.item-info:before { color }`)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit c00eac3 into rust-lang:master Dec 15, 2022
@rustbot rustbot added this to the 1.68.0 milestone Dec 15, 2022
@ComputerDruid ComputerDruid deleted the async_check branch December 15, 2022 22:46
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 14, 2023
…jackh726

Stabilize `async fn` and return-position `impl Trait` in trait

# Stabilization report

This report proposes the stabilization of `#![feature(return_position_impl_trait_in_trait)]` ([RPITIT][RFC 3425]) and `#![feature(async_fn_in_trait)]` ([AFIT][RFC 3185]). These are both long awaited features that increase the expressiveness of the Rust language and trait system.

Closes rust-lang#91611

[RFC 3185]: https://rust-lang.github.io/rfcs/3185-static-async-fn-in-trait.html
[RFC 3425]: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html

## Updates from thread

The thread has covered two major concerns:

* [Given that we don't have RTN, what should we stabilize?](rust-lang#115822 (comment)) -- proposed resolution is [adding a lint](rust-lang#115822 (comment)) and [careful messaging](rust-lang#115822 (comment))
* [Interaction between outlives bounds and capture semantics](rust-lang#115822 (comment)) -- This is fixable in a forwards-compatible way via rust-lang#116040, and also eventually via ATPIT.

## Stabilization Summary

This stabilization allows the following examples to work.

### Example of return-position `impl Trait` in trait definition

```rust
trait Bar {
    fn bar(self) -> impl Send;
}
```

This declares a trait method that returns *some* type that implements `Send`.  It's similar to writing the following using an associated type, except that the associated type is anonymous.

```rust
trait Bar {
    type _0: Send;
    fn bar(self) -> Self::_0;
}
```

### Example of return-position `impl Trait` in trait implementation

```rust
impl Bar for () {
    fn bar(self) -> impl Send {}
}
```

This defines a method implementation that returns an opaque type, just like [RPIT][RFC 1522] does, except that all in-scope lifetimes are captured in the opaque type (as is already true for `async fn` and as is expected to be true for RPIT in Rust Edition 2024), as described below.

[RFC 1522]: https://rust-lang.github.io/rfcs/1522-conservative-impl-trait.html

### Example of `async fn` in trait

```rust
trait Bar {
    async fn bar(self);
}

impl Bar for () {
    async fn bar(self) {}
}
```

This declares a trait method that returns *some* [`Future`](https://doc.rust-lang.org/core/future/trait.Future.html) and a corresponding method implementation.  This is equivalent to writing the following using RPITIT.

```rust
use core::future::Future;

trait Bar {
    fn bar(self) -> impl Future<Output = ()>;
}

impl Bar for () {
    fn bar(self) -> impl Future<Output = ()> { async {} }
}
```

The desirability of this desugaring being available is part of why RPITIT and AFIT are being proposed for stabilization at the same time.

## Motivation

Long ago, Rust added [RPIT][RFC 1522] and [`async`/`await`][RFC 2394].  These are major features that are widely used in the ecosystem.  However, until now, these feature could not be used in *traits* and trait implementations.  This left traits as a kind of second-class citizen of the language.  This stabilization fixes that.

[RFC 2394]: https://rust-lang.github.io/rfcs/2394-async_await.html

### `async fn` in trait

Async/await allows users to write asynchronous code much easier than they could before. However, it doesn't play nice with other core language features that make Rust the great language it is, like traits. Support for `async fn` in traits has been long anticipated and was not added before due to limitations in the compiler that have now been lifted.

`async fn` in traits will unblock a lot of work in the ecosystem and the standard library. It is not currently possible to write a trait that is implemented using `async fn`. The workarounds that exist are undesirable because they require allocation and dynamic dispatch, and any trait that uses them will become obsolete once native `async fn` in trait is stabilized.

We also have ample evidence that there is demand for this feature from the [`async-trait` crate][async-trait], which emulates the feature using dynamic dispatch. The async-trait crate is currently the rust-lang#5 async crate on crates.io ranked by recent downloads, receiving over 78M all-time downloads. According to a [recent analysis][async-trait-analysis], 4% of all crates use the `#[async_trait]` macro it provides, representing 7% of all function and method signatures in trait definitions on crates.io. We think this is a *lower bound* on demand for the feature, because users are unlikely to use `#[async_trait]` on public traits on crates.io for the reasons already given.

[async-trait]: https://crates.io/crates/async-trait
[async-trait-analysis]: https://rust-lang.zulipchat.com/#narrow/stream/315482-t-compiler.2Fetc.2Fopaque-types/topic/RPIT.20capture.20rules.20.28capturing.20everything.29/near/389496292

### Return-position `impl Trait` in trait

`async fn` always desugars to a function that returns `impl Future`.

```rust!
async fn foo() -> i32 { 100 }

// Equivalent to:
fn foo() -> impl Future<Output = i32> { async { 100 } }
```

All `async fn`s today can be rewritten this way. This is useful because it allows adding behavior that runs at the time of the function call, before the first `.await` on the returned future.

In the spirit of supporting the same set of features on `async fn` in traits that we do outside of traits, it makes sense to stabilize this as well. As described by the [RPITIT RFC][rpitit-rfc], this includes the ability to mix and match the equivalent forms in traits and their corresponding impls:

```rust!
trait Foo {
    async fn foo(self) -> i32;
}

// Can be implemented as:
impl Foo for MyType {
    fn foo(self) -> impl Future<Output = i32> {
        async { 100 }
    }
}
```

Return-position `impl Trait` in trait is useful for cases beyond async, just as regular RPIT is. As a simple example, the RFC showed an alternative way of writing the `IntoIterator` trait with one fewer associated type.

```rust!
trait NewIntoIterator {
    type Item;
    fn new_into_iter(self) -> impl Iterator<Item = Self::Item>;
}

impl<T> NewIntoIterator for Vec<T> {
    type Item = T;
    fn new_into_iter(self) -> impl Iterator<Item = T> {
        self.into_iter()
    }
}
```

[rpitit-rfc]: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html

## Major design decisions

This section describes the major design decisions that were reached after the RFC was accepted:

- EDIT: Lint against async fn in trait definitions

    - Until the [send bound problem](https://smallcultfollowing.com/babysteps/blog/2023/02/01/async-trait-send-bounds-part-1-intro/) is resolved, the use of `async fn` in trait definitions could lead to a bad experience for people using work-stealing executors (by far the most popular choice). However, there are significant use cases for which the current support is all that is needed (single-threaded executors, such as those used in embedded use cases, as well as thread-per-core setups). We are prioritizing serving users well over protecting people from misuse, and therefore, we opt to stabilize the full range of functionality; however, to help steer people correctly, we are will issue a warning on the use of `async fn` in trait definitions that advises users about the limitations. (See [this summary comment](rust-lang#115822 (comment)) for the details of the concern, and [this comment](rust-lang#115822 (comment)) for more details about the reasoning that led to this conclusion.)

- Capture rules:

    - The RFC's initial capture rules for lifetimes in impls/traits were found to be imprecisely precise and to introduce various inconsistencies. After much discussion, the decision was reached to make `-> impl Trait` in traits/impls capture *all* in-scope parameters, including both lifetimes and types. This is a departure from the behavior of RPITs in other contexts; an RFC is currently being authored to change the behavior of RPITs in other contexts in a future edition.

    - Major discussion links:

        - [Lang team design meeting from 2023-07-26](https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ?view)

- Refinement:

    - The [refinement RFC] initially proposed that impl signatures that are more specific than their trait are not allowed unless the `#[refine]` attribute was included, but left it as an open question how to implement this. The stabilized proposal is that it is not a hard error to omit `#[refine]`, but there is a lint which fires if the impl's return type is more precise than the trait. This greatly simplified the desugaring and implementation while still achieving the original goal of ensuring that users do not accidentally commit to a more specific return type than they intended.

    - Major discussion links:

        - [Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/.60.23.5Brefine.5D.60.20as.20a.20lint)

[refinement RFC]: https://rust-lang.github.io/rfcs/3245-refined-impls.html

## What is stabilized

### Async functions in traits and trait implementations

* `async fn` are now supported in traits and trait implementations.
* Associated functions in traits that are `async` may have default bodies.

### Return-position impl trait in traits and trait implementations

* Return-position `impl Trait`s are now supported in traits and trait implementations.
    * Return-position `impl Trait` in implementations are treated like regular return-position `impl Trait`s, and therefore behave according to the same inference rules for hidden type inference and well-formedness.
* Associated functions in traits that name return-position `impl Trait`s may have default bodies.
* Implementations may provide either concrete types or `impl Trait` for each corresponding `impl Trait` in the trait method signature.

For a detailed exploration of the technical implementation of return-position `impl Trait` in traits, see [the dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html).

### Mixing `async fn` in trait and return-position `impl Trait` in trait

A trait function declaration that is `async fn ..() -> T` may be satisfied by an implementation function that returns `impl Future<Output = T>`, or vice versa.

```rust
trait Async {
    async fn hello();
}

impl Async for () {
    fn hello() -> impl Future<Output = ()> {
        async {}
    }
}

trait RPIT {
    fn hello() -> impl Future<Output = String>;
}

impl RPIT for () {
    async fn hello() -> String {
        "hello".to_string()
    }
}
```

### Return-position `impl Trait` in traits and trait implementations capture all in-scope lifetimes

Described above in "major design decisions".

### Return-position `impl Trait` in traits are "always revealing"

When a trait uses `-> impl Trait` in return position, it logically desugars to an associated type that represents the return (the actual implementation in the compiler is different, as described below). The value of this associated type is determined by the actual return type written in the impl; if the impl also uses `-> impl Trait` as the return type, then the value of the associated type is an opaque type scoped to the impl method (similar to what you would get when calling an inherent function returning `-> impl Trait`). As with any associated type, the value of this special associated type can be revealed by the compiler if the compiler can figure out what impl is being used.

For example, given this trait:

```rust
trait AsDebug {
    fn as_debug(&self) -> impl Debug;
}
```

A function working with the trait generically is only able to see that the return value is `Debug`:

```rust
fn foo<T: AsDebug>(t: &T) {
    let u = t.as_debug();
    println!("{}", u); // ERROR: `u` is not known to implement `Display`
}
```

But if a function calls `as_debug` on a known type (say, `u32`), it may be able to resolve the return type more specifically, if that implementation specifies a concrete type as well:

```rust
impl AsDebug for u32 {
    fn as_debug(&self) -> u32 {
        *self
    }
}

fn foo(t: &u32) {
    let u: u32 = t.as_debug(); // OK!
    println!("{}",  t.as_debug()); // ALSO OK (since `u32: Display`).
}
```

The return type used in the impl therefore represents a **semver binding** promise from the impl author that the return type of `<u32 as AsDebug>::as_debug` will not change. This could come as a surprise to users, who might expect that they are free to change the return type to any other type that implements `Debug`. To address this, we include a [`refining_impl_trait` lint](rust-lang#115582) that warns if the impl uses a specific type -- the `impl AsDebug for u32` above, for example, would toggle the lint.

The lint message explains what is going on and encourages users to `allow` the lint to indicate that they meant to refine the return type:

```rust
impl AsDebug for u32 {
    #[allow(refining_impl_trait)]
    fn as_debug(&self) -> u32 {
        *self
    }
}
```

[RFC rust-lang#3245](rust-lang/rfcs#3245) proposed a new attribute, `#[refine]`, that could also be used to "opt-in" to refinements like this (and which would then silence the lint). That RFC is not currently implemented -- the `#[refine]` attribute is also expected to reveal other details from the signature and has not yet been fully implemented.

### Return-position `impl Trait` and `async fn` in traits are opted-out of object safety checks when the parent function has `Self: Sized`

```rust
trait IsObjectSafe {
    fn rpit() -> impl Sized where Self: Sized;
    async fn afit() where Self: Sized;
}
```

Traits that mention return-position `impl Trait` or `async fn` in trait when the associated function includes a `Self: Sized` bound will remain object safe. That is because the associated function that defines them will be opted-out of the vtable of the trait, and the associated types will be unnameable from any trait object.

This can alternatively be seen as a consequence of rust-lang#112319 (comment) and the desugaring of return-position `impl Trait` in traits to associated types which inherit the where-clauses of the associated function that defines them.

## What isn't stabilized (aka, potential future work)

### Dynamic dispatch

As stabilized, traits containing RPITIT and AFIT are **not dyn compatible**. This means that you cannot create `dyn Trait` objects from them and can only use static dispatch. The reason for this limitation is that dynamic dispatch support for RPITIT and AFIT is more complex than static dispatch, as described on the [async fundamentals page](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/challenges/dyn_traits.html). The primary challenge to using `dyn Trait` in today's Rust is that **`dyn Trait` today must list the values of all associated types**. This means you would have to write `dyn for<'s> Trait<Foo<'s> = XXX>` where `XXX` is the future type defined by the impl, such as `F_A`. This is not only verbose (or impossible), it also uniquely ties the `dyn Trait` to a particular impl, defeating the whole point of `dyn Trait`.

The precise design for handling dynamic dispatch is not yet determined. Top candidates include:

- [callee site selection][], in which we permit unsized return values so that the return type for an `-> impl Foo` method be can be `dyn Foo`, but then users must specify the type of wide pointer at the call-site in some fashion.

- [`dyn*`][], where we create a built-in encapsulation of a "wide pointer" and map the associated type corresponding to an RPITIT to the corresponding `dyn*` type (`dyn*` itself is not exposed to users as a type in this proposal, though that could be a future extension).

[callee site selection]: https://smallcultfollowing.com/babysteps/blog/2022/09/21/dyn-async-traits-part-9-callee-site-selection/

[`dyn*`]: https://smallcultfollowing.com/babysteps/blog/2022/03/29/dyn-can-we-make-dyn-sized/

### Where-clause bounds on return-position `impl Trait` in traits or async futures (RTN/ART)

One limitation of async fn in traits and RPITIT as stabilized is that there is no way for users to write code that adds additional bounds beyond those listed in the `-> impl Trait`. The most common example is wanting to write a generic function that requires that the future returned from an `async fn` be `Send`:

```rust
trait Greet {
    async fn greet(&self);
}

fn greet_in_parallel<G: Greet>(g: &G) {
    runtime::spawn(async move {
        g.greet().await; //~ ERROR: future returned by `greet` may not be `Send`
    })
}
```

Currently, since the associated types added for the return type are anonymous, there is no where-clause that could be added to make this code compile.

There have been various proposals for how to address this problem (e.g., [return type notation][rtn] or having an annotation to give a name to the associated type), but we leave the selection of one of those mechanisms to future work.

[rtn]: https://smallcultfollowing.com/babysteps/blog/2023/02/13/return-type-notation-send-bounds-part-2/

In the meantime, there are workarounds that one can use to address this problem, listed below.

#### Require all futures to be `Send`

For many users, the trait may only ever be used with `Send` futures, in which case one can write an explicit `impl Future + Send`:

```rust
trait Greet {
    fn greet(&self) -> impl Future<Output = ()> + Send;
}
```

The nice thing about this is that it is still compatible with using `async fn` in the trait impl. In the async working group case studies, we found that this could work for the [builder provider API](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/builder-provider-api.html). This is also the default approach used by the `#[async_trait]` crate which, as we have noted, has seen widespread adoption.

#### Avoid generics

This problem only applies when the `Self` type is generic. If the `Self` type is known, then the precise return type from an `async fn` is revealed, and the `Send` bound can be inferred thanks to auto-trait leakage. Even in cases where generics may appear to be required, it is sometimes possible to rewrite the code to avoid them. The [socket handler refactor](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/socket-handler.html) case study provides one such example.

### Unify capture behavior for `-> impl Trait` in inherent methods and traits

As stabilized, the capture behavior for `-> impl Trait` in a trait (whether as part of an async fn or a RPITIT) captures all types and lifetimes, whereas the existing behavior for inherent methods only captures types and lifetimes that are explicitly referenced. Capturing all lifetimes in traits was necessary to avoid various surprising inconsistencies; the expressed intent of the lang team is to extend that behavior so that we also capture all lifetimes in inherent methods, which would create more consistency and also address a common source of user confusion, but that will have to happen over the 2024 edition. The RFC is in progress. Should we opt not to accept that RFC, we can bring the capture behavior for `-> impl Trait` into alignment in other ways as part of the 2024 edition.

### `impl_trait_projections`

Orthgonal to `async_fn_in_trait` and `return_position_impl_trait_in_trait`, since it can be triggered on stable code. This will be stabilized separately in [rust-lang#115659](rust-lang#115659).

<details>
If we try to write this code without `impl_trait_projections`, we will get an error:

```rust
#![feature(async_fn_in_trait)]

trait Foo {
    type Error;
    async fn foo(&mut self) -> Result<(), Self::Error>;
}

impl<T: Foo> Foo for &mut T {
    type Error = T::Error;
    async fn foo(&mut self) -> Result<(), Self::Error> {
        T::foo(self).await
    }
}
```

The error relates to the use of `Self` in a trait impl when the self type has a lifetime. It can be worked around by rewriting the impl not to use `Self`:

```rust
#![feature(async_fn_in_trait)]

trait Foo {
    type Error;
    async fn foo(&mut self) -> Result<(), Self::Error>;
}

impl<T: Foo> Foo for &mut T {
    type Error = T::Error;
    async fn foo(&mut self) -> Result<(), <&mut T as Foo>::Error> {
        T::foo(self).await
    }
}
```
</details>

## Tests

Tests are generally organized between return-position `impl Trait` and `async fn` in trait, when the distinction matters.
* RPITIT: https://github.com/rust-lang/rust/tree/master/tests/ui/impl-trait/in-trait
* AFIT: https://github.com/rust-lang/rust/tree/master/tests/ui/async-await/in-trait

## Remaining bugs and open issues

* rust-lang#112047: Indirection introduced by `async fn` and return-position `impl Trait` in traits may hide cycles in opaque types, causing overflow errors that can only be discovered by monomorphization.
* rust-lang#111105 - `async fn` in trait is susceptible to issues with checking auto traits on futures' generators, like regular `async`. This is a manifestation of rust-lang#110338.
    * This was deemed not blocking because fixing it is forwards-compatible, and regular `async` is subject to the same issues.
* rust-lang#104689: `async fn` and return-position `impl Trait` in trait requires the late-bound lifetimes in a trait and impl function signature to be equal.
    * This can be relaxed in the future with a smarter lexical region resolution algorithm.
* rust-lang#102527: Nesting return-position `impl Trait` in trait deeply may result in slow compile times.
    * This has only been reported once, and can be fixed in the future.
* rust-lang#108362: Inference between return types and generics of a function may have difficulties when there's an `.await`.
    * This isn't related to AFIT (rust-lang#108362 (comment)) -- using traits does mean that there's possibly easier ways to hit it.
* rust-lang#112626: Because `async fn` and return-position `impl Trait` in traits lower to associated types, users may encounter strange behaviors when implementing circularly dependent traits.
    * This is not specific to RPITIT, and is a limitation of associated types: rust-lang#112626 (comment)
* **(Nightly)** rust-lang#108309: `async fn` and return-position `impl Trait` in trait do not support specialization. This was deemed not blocking, since it can be fixed in the future (e.g. rust-lang#108321) and specialization is a nightly feature.

#### (Nightly) Return type notation bugs

RTN is not being stabilized here, but there are some interesting outstanding bugs. None of them are blockers for AFIT/RPITIT, but I'm noting them for completeness.

<details>

* rust-lang#109924 is a bug that occurs when a higher-ranked trait bound has both inference variables and associated types. This is pre-existing -- RTN just gives you a more convenient way of producing them. This should be fixed by the new trait solver.
* rust-lang#109924 is a manifestation of a more general issue with `async` and auto-trait bounds: rust-lang#110338. RTN does not cause this issue, just allows us to put `Send` bounds on the anonymous futures that we have in traits.
* rust-lang#112569 is a bug similar to associated type bounds, where nested bounds are not implied correctly.

</details>

## Alternatives

### Do nothing

We could choose not to stabilize these features. Users that can use the `#[async_trait]` macro would continue to do so. Library maintainers would continue to avoid async functions in traits, potentially blocking the stable release of many useful crates.

### Stabilize `impl Trait` in associated type instead

AFIT and RPITIT solve the problem of returning unnameable types from trait methods. It is also possible to solve this by using another unstable feature, `impl Trait` in an associated type. Users would need to define an associated type in both the trait and trait impl:

```rust!
trait Foo {
    type Fut<'a>: Future<Output = i32> where Self: 'a;
    fn foo(&self) -> Self::Fut<'_>;
}

impl Foo for MyType {
    type Fut<'a> where Self: 'a = impl Future<Output = i32>;
    fn foo(&self) -> Self::Fut<'_> {
        async { 42 }
    }
}
```

This also has the advantage of allowing generic code to bound the associated type. However, it is substantially less ergonomic than either `async fn` or `-> impl Future`, and users still expect to be able to use those features in traits. **Even if this feature were stable, we would still want to stabilize AFIT and RPITIT.**

That said, we can have both. `impl Trait` in associated types is desireable because it can be used in existing traits with explicit associated types, among other reasons. We *should* stabilize this feature once it is ready, but that's outside the scope of this proposal.

### Use the old capture semantics for RPITIT

We could choose to make the capture rules for RPITIT consistent with the existing rules for RPIT. However, there was strong consensus in a recent [lang team meeting](https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ?view) that we should *change* these rules, and furthermore that new features should adopt the new rules.

This is consistent with the tenet in RFC 3085 of favoring ["Uniform behavior across editions"](https://rust-lang.github.io/rfcs/3085-edition-2021.html#uniform-behavior-across-editions) when possible. It greatly reduces the complexity of the feature by not requiring us to answer, or implement, the design questions that arise out of the interaction between the current capture rules and traits. This reduction in complexity – and eventual technical debt – is exactly in line with the motivation listed in the aforementioned RFC.

### Make refinement a hard error

Refinement (`refining_impl_trait`) is only a concern for library authors, and therefore doesn't really warrant making into a deny-by-default warning or an error.

Additionally, refinement is currently checked via a lint that compares bounds in the `impl Trait`s in the trait and impl syntactically. This is good enough for a warning that can be opted-out, but not if this were a hard error, which would ideally be implemented using fully semantic, implicational logic. This was implemented (rust-lang#111931), but also is an unnecessary burden on the type system for little pay-off.

## History

- Dec 7, 2021: [RFC rust-lang#3185: Static async fn in traits](https://rust-lang.github.io/rfcs/3185-static-async-fn-in-trait.html) merged
- Sep 9, 2022: [Initial implementation](rust-lang#101224) of AFIT and RPITIT landed
- Jun 13, 2023: [RFC rust-lang#3425: Return position `impl Trait` in traits](https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html) merged

<!--These will render pretty when pasted into github-->
Non-exhaustive list of PRs that are particularly relevant to the implementation:

- rust-lang#101224
- rust-lang#103491
- rust-lang#104592
- rust-lang#108141
- rust-lang#108319
- rust-lang#108672
- rust-lang#112988
- rust-lang#113182 (later made redundant by rust-lang#114489)
- rust-lang#113215
- rust-lang#114489
- rust-lang#115467
- rust-lang#115582

Doc co-authored by `@nikomatsakis,` `@tmandry,` `@traviscross.` Thanks also to `@spastorino,` `@cjgillot` (for changes to opaque captures!), `@oli-obk` for many reviews, and many other contributors and issue-filers. Apologies if I left your name off 😺
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-translation Area: Translation infrastructure, and migrating existing diagnostics to SessionDiagnostic S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

async fn in trait should require async fn in impl
9 participants