You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Inconsistency in Send/Sync requirements for async/await #59245
The following code, when switching between lines 14 and 15 (the two versions of the for line), oscillates between compiling and not compiling because it requires Sync for the dyn Send items:
I think this is working as intended-- I'd expect that in order for the future to be Send, all references held across await! points must be Send, which means the data they reference must be Sync. In the for loop here, a reference to v is being held across the await!. cc @nikomatsakis for another unfortunate side-effect of temporary references becoming extraordinarily visible under async/await!.
I guess it's also worth pointing out that what isn't "working as intended" here is that the reference in question shouldn't need to be held across the await!, since it's only being used to produce a new temporary (the range).
Actually, if the reference points to inside the future itself, I think that the pointed-to stuff might only need to be Send, as there would at any point in time be only a single thread referring to them. But that is another issue (or another feature request, actually -- and I'm not even sure yet it'd be sound), and here what I wanted to point out was that the temporary borrow is propagated further than would be expected.
I've marked this as deferred because of how #57017 is triaged, but I tend to think we should take some action here -- at least trying to get more precise.
question: If others agree this is a dup, maybe we can close this issue? (Perhaps move the examples over to #57017)
I'm not sure this is the same as #57017, but it does look similar -- will let someone more knowledgeable than me about compiler internals choose whether to close this :)
It seems like #57017 is more specifically about a code pattern that leads to this happening, the underlying issue here is just that a generator holding a self-reference across a yield point requires the referenced object to be Sync when it shouldn't, it just needs to be Send as the reference will move threads along with the referenced object. #57017 shows one possible way to cause this issue, because computing the live variables across the yield point inside a match includes the matched expression when it is unnecessary.
#![feature(async_await)]asyncfnfoo(x:Box<dynSend>){asyncfnbar(){}let x = &x;bar().await;}fnassert_send<T:Send>(_:T){}fnmain(){assert_send(foo(Box::new(5)));}
error[E0277]: `dyn std::marker::Send` cannot be shared between threads safely
--> src/main.rs:12:5
|
12 | assert_send(foo(Box::new(5)));
| ^^^^^^^^^^^ `dyn std::marker::Send` cannot be shared between threads safely
|
= help: the trait `std::marker::Sync` is not implemented for `dyn std::marker::Send`
= note: required because of the requirements on the impl of `std::marker::Sync` for `std::ptr::Unique<dyn std::marker::Send>`
= note: required because it appears within the type `std::boxed::Box<dyn std::marker::Send>`
= note: required because of the requirements on the impl of `std::marker::Send` for `&std::boxed::Box<dyn std::marker::Send>`
= note: required because it appears within the type `for<'r, 's, 't0> {std::boxed::Box<(dyn std::marker::Send + 'r)>, &'s std::boxed::Box<(dyn std::marker::Send + 't0)>, impl std::future::Future, ()}`
= note: required because it appears within the type `[static generator@src/main.rs:3:32: 7:2 x:std::boxed::Box<(dyn std::marker::Send + 'static)> for<'r, 's, 't0> {std::boxed::Box<(dyn std::marker::Send + 'r)>, &'s std::boxed::Box<(dyn std::marker::Send + 't0)>, impl std::future::Future, ()}]`
= note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:3:32: 7:2 x:std::boxed::Box<(dyn std::marker::Send + 'static)> for<'r, 's, 't0> {std::boxed::Box<(dyn std::marker::Send + 'r)>, &'s std::boxed::Box<(dyn std::marker::Send + 't0)>, impl std::future::Future, ()}]>`
= note: required because it appears within the type `impl std::future::Future`
= note: required because it appears within the type `impl std::future::Future`
note: required by `assert_send`
--> src/main.rs:9:1
|
9 | fn assert_send<T: Send>(_: T) {}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Given that #57017 appears to be in the process of living again, maybe this should be made live again? (note: I have literally no idea how labels work for async/await)
This results in wartiness, and currently misleading compiler help, when structuring generic async code:
use std::fmt::Display;asyncfnrun(mutstate:implDisplay){do_stuff(&state).await;// ...}asyncfndo_stuff(state:&implDisplay){println!("{state}");}fnspawn_task<T>(state:T)whereT:Display + Send + 'static,{
tokio::spawn(run(state));}
The compiler produces an error and further suggests:
help: consider further restricting this bound
|
14 | T: Display + Send + 'static + std::marker::Sync,
| +++++++++++++++++++
The non-restrictive solution is to make the reference passed to do_stuff mutable (i.e. provably exclusive), even though mutability is not required by the function body.
If a coroutine only captures Send types, and only has Send types + references to Send types through each yield-point, it should be Send. The references across the yield points are derived from captures, owned (self-referential) values, and globals. Captures must be Send; Globals must be Send. It boils down to whether !Send values are alive across yield points, excluding references to !Sync values.
Probably needs a new auto trait to opt out references, specifically used by coroutines. Something like ThreadUnpin which is impl for all Send types and for references to ThreadUnpin + !Sync types.
I hit this issue today with a huge chunk of async code. I am not looking forward to refactoring it.
If you have further thoughts about that, they should probably just go there.
In terms of filing it as an ask that it "just work" somehow, I don't personally see how that could ever be supported in general. It's probably just worth tracking it in #129105 for now, unless you see some smaller special case that could conceivably be supported and want to file that separately.
Activity
cramertj commentedon Mar 18, 2019
I think this is working as intended-- I'd expect that in order for the future to be
Send, all references held acrossawait!points must beSend, which means the data they reference must beSync. In theforloop here, a reference tovis being held across theawait!. cc @nikomatsakis for another unfortunate side-effect of temporary references becoming extraordinarily visible under async/await!.cramertj commentedon Mar 18, 2019
I guess it's also worth pointing out that what isn't "working as intended" here is that the reference in question shouldn't need to be held across the
await!, since it's only being used to produce a new temporary (the range).Ekleog commentedon Mar 18, 2019
Actually, if the reference points to inside the future itself, I think that the pointed-to stuff might only need to be
Send, as there would at any point in time be only a single thread referring to them. But that is another issue (or another feature request, actually -- and I'm not even sure yet it'd be sound), and here what I wanted to point out was that the temporary borrow is propagated further than would be expected.earthengine commentedon Mar 18, 2019
FYI, I was posted in internals for a similar issue regarding pattern matches.
Playground
nikomatsakis commentedon Apr 16, 2019
I believe this is a duplicate of #57017.
nikomatsakis commentedon Apr 16, 2019
I've marked this as deferred because of how #57017 is triaged, but I tend to think we should take some action here -- at least trying to get more precise.
question: If others agree this is a dup, maybe we can close this issue? (Perhaps move the examples over to #57017)
Ekleog commentedon Apr 16, 2019
I'm not sure this is the same as #57017, but it does look similar -- will let someone more knowledgeable than me about compiler internals choose whether to close this :)
Nemo157 commentedon Jul 22, 2019
It seems like #57017 is more specifically about a code pattern that leads to this happening, the underlying issue here is just that a generator holding a self-reference across a yield point requires the referenced object to be
Syncwhen it shouldn't, it just needs to beSendas the reference will move threads along with the referenced object. #57017 shows one possible way to cause this issue, because computing the live variables across the yield point inside a match includes the matched expression when it is unnecessary.Nemo157 commentedon Jul 22, 2019
Minimised example (playground):
Ekleog commentedon Jun 29, 2020
I've just hit this again, with this:
(playground link)
Given that #57017 appears to be in the process of living again, maybe this should be made live again? (note: I have literally no idea how labels work for async/await)
mzabaluev commentedon Aug 14, 2024
This results in wartiness, and currently misleading compiler help, when structuring generic async code:
The compiler produces an error and further suggests:
The non-restrictive solution is to make the reference passed to
do_stuffmutable (i.e. provably exclusive), even though mutability is not required by the function body.ds84182 commentedon Oct 30, 2024
If a coroutine only captures
Sendtypes, and only hasSendtypes + references toSendtypes through each yield-point, it should beSend. The references across the yield points are derived from captures, owned (self-referential) values, and globals. Captures must beSend; Globals must beSend. It boils down to whether!Sendvalues are alive across yield points, excluding references to!Syncvalues.Probably needs a new auto trait to opt out references, specifically used by coroutines. Something like
ThreadUnpinwhich is impl for allSendtypes and for references toThreadUnpin + !Synctypes.I hit this issue today with a huge chunk of async code. I am not looking forward to refactoring it.
traviscross commentedon Oct 30, 2024
The originally-reported example in this issue now works, as do the later minimizations. So I'm going to close this issue, as it seems resolved.
If you're still hitting an issue, it may be something different, so please file a new issue and provide a minimized example if possible.
mzabaluev commentedon Nov 2, 2024
My example still fails as of rustc 1.84.0-nightly (705cfe0 2024-11-01). Should I file a separate issue?
traviscross commentedon Nov 3, 2024
We discussed adding a diagnostic for that over in:
Syncbound when shareable reference is passed across or into await #129105If you have further thoughts about that, they should probably just go there.
In terms of filing it as an ask that it "just work" somehow, I don't personally see how that could ever be supported in general. It's probably just worth tracking it in #129105 for now, unless you see some smaller special case that could conceivably be supported and want to file that separately.