Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with GATs, async, and Send-bounds #90696

Open
Tracked by #110338
JanBeh opened this issue Nov 8, 2021 · 12 comments
Open
Tracked by #110338

Problem with GATs, async, and Send-bounds #90696

JanBeh opened this issue Nov 8, 2021 · 12 comments
Labels
A-associated-items Area: Associated items (types, constants & functions) A-async-await Area: Async & Await A-GATs Area: Generic associated types (GATs) A-lifetimes Area: Lifetimes / regions A-traits Area: Trait system AsyncAwait-Triaged Async-await issues that have been triaged during a working group meeting. GATs-triaged Issues using the `generic_associated_types` feature that have been triaged requires-nightly This issue requires a nightly compiler in some way. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@JanBeh
Copy link
Contributor

JanBeh commented Nov 8, 2021

I have experienced problems with Send bounds on GATs. The original code where the problem occurred uses the async-trait crate and looks as follows:

#![feature(generic_associated_types)]

use async_trait::async_trait;
use std::ops::Deref;

async fn some_async_task() {}

#[async_trait]
trait Source {
    type T;
    type Wrapper<'a>: Deref<Target = Self::T> + Send
    where
        Self: 'a;
    async fn retrieve(&mut self) -> Self::Wrapper<'_>;
}

struct S {
    state: i32,
}

impl S {
    fn new() -> Self {
        S { state: 0 }
    }
}

#[async_trait]
impl Source for S {
    type T = i32;
    type Wrapper<'a> = &'a Self::T;
    async fn retrieve(&mut self) -> Self::Wrapper<'_> {
        self.state += 1;
        &self.state
    }
}

#[async_trait]
trait User {
    // After updating nightly rustc, this won't work anymore:
    async fn process<'a, 'b, S>(
        &'a self,
        source: &'b mut S,
    ) -> S::Wrapper<'b>
    where
        S: Source + Send,
    {
        let result = source.retrieve().await;
        some_async_task().await;
        result
    }
}

struct U {}

#[async_trait]
impl User for U {}

#[tokio::main]
async fn main() {
    let mut s = S::new();
    let u = U {};
    let value: &i32 = u.process(&mut s).await;
    println!("Result = {}", value);
}

Using the Rust playground with Nightly version: 1.58.0-nightly (2021-11-07 46b8e74), I get:

   Compiling playground v0.0.1 (/playground)
error[E0311]: the parameter type `S` may not live long enough
  --> src/main.rs:46:5
   |
40 |       async fn process<'a, 'b, S>(
   |                                - help: consider adding an explicit lifetime bound...: `S: 'c`
...
46 | /     {
47 | |         let result = source.retrieve().await;
48 | |         some_async_task().await;
49 | |         result
50 | |     }
   | |_____^ ...so that the type `S` will meet its required lifetime bounds

error: could not compile `playground` due to previous error

If I follow the compiler's advice literally, then of course I'll get:

error[E0261]: use of undeclared lifetime name `'c`

If I declare 'c as a lifetime parameter, then the compiler demands I should add a bound S: 'd, and so on.

This code worked with some earlier version of nightly Rust (without the where Self: 'a bound, and before where Self: 'a was required in the GAT). Unfortunately I do not remember which version that was (a few weeks ago).

However, even before the update of rustc, I had difficulties to remove the Send bound from the GATs definition and to include a bound in another method that uses the GATs. This problem still persists and can be demonstrated with the following code:

#![feature(generic_associated_types)]

use async_trait::async_trait;
use std::ops::Deref;

async fn some_async_task() {}

#[async_trait]
trait Source {
    type T;
    // I removed the `Send` bound here:
    type Wrapper<'a>: Deref<Target = Self::T>
    where
        Self: 'a;
    async fn retrieve(&mut self) -> Self::Wrapper<'_>;
}

struct S {
    state: i32,
}

impl S {
    fn new() -> Self {
        S { state: 0 }
    }
}

#[async_trait]
impl Source for S {
    type T = i32;
    type Wrapper<'a> = &'a Self::T;
    async fn retrieve(&mut self) -> Self::Wrapper<'_> {
        self.state += 1;
        &self.state
    }
}

#[async_trait]
trait User {
    async fn process<'a, 'b, S>(
        &'a self,
        source: &'b mut S,
    ) -> S::Wrapper<'b>
    where
        S: Source + Send + 'static,
        // And added it here:
        <S as Source>::Wrapper<'b>: Send,
    {
        let result = source.retrieve().await;
        some_async_task().await;
        result
    }
}

struct U {}

#[async_trait]
impl User for U {}

#[tokio::main]
async fn main() {
    let mut s = S::new();
    let u = U {};
    let value: &i32 = u.process(&mut s).await;
    println!("Result = {}", value);
}

Using again Rust playground with Nightly version: 1.58.0-nightly (2021-11-07 46b8e74), I get:

   Compiling playground v0.0.1 (/playground)
error: implementation of `Send` is not general enough
  --> src/main.rs:48:5
   |
48 | /     {
49 | |         let result = source.retrieve().await;
50 | |         some_async_task().await;
51 | |         result
52 | |     }
   | |_____^ implementation of `Send` is not general enough
   |
   = note: `<S as Source>::Wrapper<'0>` must implement `Send`, for any lifetime `'0`...
   = note: ...but `Send` is actually implemented for the type `<S as Source>::Wrapper<'b>`

error: could not compile `playground` due to previous error

If I try to use a HRTB (for<'z> <S as Source>::Wrapper<'z>: Send), then I get:

   Compiling playground v0.0.1 (/playground)
error[E0277]: `<_ as Source>::Wrapper<'z>` cannot be sent between threads safely
  --> src/main.rs:64:25
   |
64 |     let value: &i32 = u.process(&mut s).await;
   |                         ^^^^^^^ `<_ as Source>::Wrapper<'z>` cannot be sent between threads safely
   |
   = help: the trait `for<'z> Send` is not implemented for `<_ as Source>::Wrapper<'z>`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `playground` due to previous error

I'm not sure if these problems are related, but there seem to be a link. There has been a discussion on the Rust users forum where another smaller example (without async-trait macros) was created:

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(async move {
        let result: PhantomData<S::Associated<'a>> = PhantomData;
        async {}.await;
    });
}

This results (using the same compiler version) in:

   Compiling playground v0.0.1 (/playground)
error[E0311]: the parameter type `S` may not live long enough
  --> src/lib.rs:18:5
   |
17 | fn foo<'a, S: Trait + 'a>() {
   |            -- help: consider adding an explicit lifetime bound...: `S: 'b +`
18 |     future::<'a, S, _>(async move {
   |     ^^^^^^^^^^^^^^^^^^ ...so that the type `S` will meet its required lifetime bounds...
   |
note: ...that is required by this bound
  --> src/lib.rs:12:30
   |
12 |     F: Future<Output = ()> + Send,
   |                              ^^^^

error: could not compile `playground` due to previous error

If I replace Send with SomeTrait with impl<T> SomeTrait for T {}, the code will compile. We guessed the problem might be auto-trait related?

See also:

@JanBeh JanBeh added the C-bug Category: This is a bug. label Nov 8, 2021
@JanBeh
Copy link
Contributor Author

JanBeh commented Nov 8, 2021

@rustbot label F-generic_associated_types

@rustbot rustbot added the F-generic_associated_types `#![feature(generic_associated_types)]` a.k.a. GATs label Nov 8, 2021
@steffahn
Copy link
Member

steffahn commented Nov 8, 2021

@rustbot label A-lifetimes, A-associated-items, A-traits, A-async-await, T-compiler, requires-nightly

@rustbot rustbot added A-associated-items Area: Associated items (types, constants & functions) A-async-await Area: Async & Await A-lifetimes Area: Lifetimes / regions A-traits Area: Trait system requires-nightly This issue requires a nightly compiler in some way. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 8, 2021
@eholk
Copy link
Contributor

eholk commented Nov 8, 2021

This looks like a tricky bug and will take some time to look into. There were some recent closure changes that might be causing this, and this sounds potentially related to #71723, #71671, and #87425.

Bisection would let us narrow down the change that caused the problem.

@rustbot label E-needs-bisection E-needs-mcve AsyncAwait-Triaged

@rustbot rustbot added AsyncAwait-Triaged Async-await issues that have been triaged during a working group meeting. E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Nov 8, 2021
@eholk
Copy link
Contributor

eholk commented Nov 8, 2021

Also potentially related: #64552, #60658.

@eholk
Copy link
Contributor

eholk commented Nov 8, 2021

Also: #70263

@steffahn
Copy link
Member

steffahn commented Nov 8, 2021

The issue already contains a minimal example,

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(async move {
        let result: PhantomData<S::Associated<'a>> = PhantomData;
        async {}.await;
    });
}

at least for one of the (potentially multiple?) issues described here.


There were some recent closure changes that might be causing this

I didn’t consider checking whether this is a regression yet. It's not necessarily related to closures though; it’s an async block, and an async fn would do the same thing.

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

async fn f<'a, S: Trait + 'a>() {
    let result: PhantomData<S::Associated<'a>> = PhantomData;
    async {}.await;
} 

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(f::<'a, S>());
}

Let me do some bisecting... okay, so it is a regression! (But not a very recent one.)

nightly-2020-10-07

warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default
  = note: see issue #44265 <https://github.com/rust-lang/rust/issues/44265> for more information

error[E0311]: the parameter type `S` may not live long enough
  --> src/main.rs:23:5
   |
22 | fn foo<'a, S: Trait + 'a>() {
   |            -- help: consider adding an explicit lifetime bound...: `S: 'b +`
23 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^
   |
   = note: the parameter type `S` must be valid for any other region...
   = note: ...so that the type `S` will meet its required lifetime bounds

error: aborting due to previous error; 1 warning emitted

nightly-2020-10-06

warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default
  = note: see issue #44265 <https://github.com/rust-lang/rust/issues/44265> for more information

warning: unused variable: `result`
  --> src/main.rs:18:9
   |
18 |     let result: PhantomData<S::Associated<'a>> = PhantomData;
   |         ^^^^^^ help: if this is intentional, prefix it with an underscore: `_result`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: function is never used: `future`
  --> src/main.rs:10:4
   |
10 | fn future<'a, S: Trait + 'a, F>(f: F) -> F
   |    ^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: function is never used: `f`
  --> src/main.rs:17:10
   |
17 | async fn f<'a, S: Trait + 'a>() {
   |          ^

warning: function is never used: `foo`
  --> src/main.rs:22:4
   |
22 | fn foo<'a, S: Trait + 'a>() {
   |    ^^^

warning: unused implementer of `Future` that must be used
  --> src/main.rs:23:5
   |
23 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_must_use)]` on by default
   = note: futures do nothing unless you `.await` or poll them

warning: 6 warnings emitted

    Finished dev [unoptimized + debuginfo] target(s) in 0.30s
********************************************************************************
Regression in nightly-2020-10-07
********************************************************************************

fetching https://static.rust-lang.org/dist/2020-10-06/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2020-10-06: 40 B / 40 B [====================================================================] 100.00 % 746.12 KB/s converted 2020-10-06 to a1dfd2490a6cb456b92e469fa550dc217e20ad6d
fetching https://static.rust-lang.org/dist/2020-10-07/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2020-10-07: 40 B / 40 B [====================================================================] 100.00 % 724.12 KB/s converted 2020-10-07 to 98edd1fbf8a68977a2a7c1312eb1ebff80515a92
looking for regression commit between 2020-10-06 and 2020-10-07
fetching (via remote github) commits from max(a1dfd2490a6cb456b92e469fa550dc217e20ad6d, 2020-10-04) to 98edd1fbf8a68977a2a7c1312eb1ebff80515a92
ending github query because we found starting sha: a1dfd2490a6cb456b92e469fa550dc217e20ad6d
get_commits_between returning commits, len: 6
  commit[0] 2020-10-05UTC: Auto merge of #77080 - richkadel:llvm-coverage-counters-2, r=tmandry
  commit[1] 2020-10-06UTC: Auto merge of #77606 - JohnTitor:rollup-7rgahdt, r=JohnTitor
  commit[2] 2020-10-06UTC: Auto merge of #77594 - timvermeulen:chain_advance_by, r=scottmcm
  commit[3] 2020-10-06UTC: Auto merge of #73905 - matthewjasper:projection-bounds-2, r=nikomatsakis
  commit[4] 2020-10-06UTC: Auto merge of #76356 - caass:hooks, r=jyn514
  commit[5] 2020-10-06UTC: Auto merge of #77386 - joshtriplett:static-glibc, r=petrochenkov
ERROR: no CI builds available between a1dfd2490a6cb456b92e469fa550dc217e20ad6d and 98edd1fbf8a68977a2a7c1312eb1ebff80515a92 within last 167 days

@rustbot label -E-needs-bisection -E-needs-mcve

@rustbot rustbot removed E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Nov 8, 2021
@steffahn
Copy link
Member

steffahn commented Nov 8, 2021

Regression most likely from

@steffahn
Copy link
Member

steffahn commented Nov 8, 2021

The issue of confusing/unnecessary HRTB-requirements is demonstrated by something like

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

async fn f<'a, S: Trait + 'a>() {
    let result: PhantomData<S::Associated<'a>> = PhantomData;
    async {}.await;
}

fn foo<'a, S: Trait + 'a>()
where
    S::Associated<'a>: Send,
{
    future::<'a, S, _>(f::<'a, S>());
}

(At least from my understanding, the S::Associated<'a>: Send is supposed to be enough.)

error: implementation of `Send` is not general enough
  --> src/main.rs:26:5
   |
26 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^ implementation of `Send` is not general enough
   |
   = note: `<S as Trait>::Associated<'0>` must implement `Send`, for any lifetime `'0`...
   = note: ...but `Send` is actually implemented for the type `<S as Trait>::Associated<'a>`

which also fails before #73905, e.g. with

warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default
  = note: see issue #44265 <https://github.com/rust-lang/rust/issues/44265> for more information

error[E0308]: mismatched types
  --> src/main.rs:26:5
   |
26 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected type `Send`
              found type `Send`

error: aborting due to previous error; 1 warning emitted

Looking further back... at some point (around nightly-2019-12-23) the error changes to

warning: the feature `generic_associated_types` is incomplete and may cause the compiler to crash
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default

error[E0109]: lifetime arguments are not allowed for this type
  --> src/main.rs:18:43
   |
18 |     let result: PhantomData<S::Associated<'a>> = PhantomData;
   |                                           ^^ lifetime argument not allowed

error[E0109]: lifetime arguments are not allowed for this type
  --> src/main.rs:24:19
   |
24 |     S::Associated<'a>: Send,
   |                   ^^ lifetime argument not allowed

error: aborting due to 2 previous errors

so there's no regression on this issue – it never worked.

@jackh726
Copy link
Member

(At least from my understanding, the S::Associated<'a>: Send is supposed to be enough.)

Changing the where clause to for<'b> S::Associated<'b>: Send makes this code compile.

@jackh726
Copy link
Member

GATs issue triage: not blocking. The examples here do use GATs, but the underlying issue seems to have roots in generality and generator capture (from what I can surmise). There also seem to be related issues that don't use GATs.

@ivan770
Copy link

ivan770 commented Jun 23, 2022

(At least from my understanding, the S::Associated<'a>: Send is supposed to be enough.)

Changing the where clause to for<'b> S::Associated<'b>: Send makes this code compile.

I should mention that third approach from this article doesn't suffer from this problem, as I was able to remove those bounds and make code compile.

@QuineDot
Copy link

QuineDot commented Jun 9, 2023

@fmease fmease added A-GATs Area: Generic associated types (GATs) and removed C-bug Category: This is a bug. F-generic_associated_types `#![feature(generic_associated_types)]` a.k.a. GATs labels Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-associated-items Area: Associated items (types, constants & functions) A-async-await Area: Async & Await A-GATs Area: Generic associated types (GATs) A-lifetimes Area: Lifetimes / regions A-traits Area: Trait system AsyncAwait-Triaged Async-await issues that have been triaged during a working group meeting. GATs-triaged Issues using the `generic_associated_types` feature that have been triaged requires-nightly This issue requires a nightly compiler in some way. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants