async/await notation for ergonomic asynchronous IO #2394
Conversation
|
||
After gaining experience & user feedback with the futures-based ecosystem, we | ||
discovered certain ergonomics challenges. Using state which needs to be shared | ||
across await points was extremely ergonomic - requiring either Arcs or join |
sfackler
Apr 6, 2018
Member
*unergonomic
*unergonomic
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
The code generated for
One of the reasons to have the closures and not the blocks is that for the closures, nobody should find it surprising that they capture their environment and variables need to outlive the function body -- that's normal for functions returning closures. For a block, though, that behavior has no precedent. |
Awesome work putting this together! I think this is a great spot to land in the design space. It might be good, on some level, to frame This framing also maps the concept of an async closure to something like a Java anonymous inner class. In that sense, an async block is actually the closer of the two concepts to existing sync closures- it's an expression that evaluates to a value (one that impls |
This comment has been hidden.
This comment has been hidden.
What is the reasoning behind using an |
A final - and extreme - alternative would be to abandon futures and async/await | ||
as the mechanism for async/await in Rust and to adopt a different paradigm. | ||
Among those suggested are a generalized effects system, monads & do notation, |
Centril
Apr 6, 2018
•
Contributor
First I want to say that I think this really well written (and readable) RFC. 🥇
I think it also lands very nicely.
In particular, my initial thinking here, is this RFC is forward-compatible with forms of effect polymorphism; which is quite nice.
One could imagine ?async
as "possibly async", i.e: the callee decides on sync or async.
And ?const
as "possibly const", -- again, the callee decides.
Then we can mix these and you get 4 different modes.
How useful all of this is an open question.
Going a bit further (or too far) effects may be try async
which could be the same as the Result-embedded futures encoding?
Also, should async fn(T) -> R
as a function type be a thing?
First I want to say that I think this really well written (and readable) RFC.
I think it also lands very nicely.
In particular, my initial thinking here, is this RFC is forward-compatible with forms of effect polymorphism; which is quite nice.
One could imagine ?async
as "possibly async", i.e: the callee decides on sync or async.
And ?const
as "possibly const", -- again, the callee decides.
Then we can mix these and you get 4 different modes.
How useful all of this is an open question.
Going a bit further (or too far) effects may be try async
which could be the same as the Result-embedded futures encoding?
Also, should async fn(T) -> R
as a function type be a thing?
RalfJung
Apr 6, 2018
Member
Also, should async fn(T) -> R as a function type a thing?
Isn't that just fn(T) -> impl Future<Item=R>
?
Also, should async fn(T) -> R as a function type a thing?
Isn't that just fn(T) -> impl Future<Item=R>
?
Centril
Apr 6, 2018
•
Contributor
That seemed to be the consensus on #rust-lang, yes =)
The question was more: should the surface syntax be a thing?
with analogy with const fn(T) -> R
.
That seemed to be the consensus on #rust-lang, yes =)
The question was more: should the surface syntax be a thing?
with analogy with const fn(T) -> R
.
Centril
Apr 6, 2018
Contributor
Some further notes on the effect polymorphism angle (still noting that this is just "hypothetical"):
We could possibly interpret async fn foo(x: T) -> R {..}
as the following:
effect<E> (E + async) fn foo(x: T) -> R {..}
.
Which universally quantifies the ambient effect E
and then you combine E + async
.
But the body of foo
can't assume anything about what E
might be, so this should also be backwards compatible in the future with such as change?
Some further notes on the effect polymorphism angle (still noting that this is just "hypothetical"):
We could possibly interpret async fn foo(x: T) -> R {..}
as the following:
effect<E> (E + async) fn foo(x: T) -> R {..}
.
Which universally quantifies the ambient effect E
and then you combine E + async
.
But the body of foo
can't assume anything about what E
might be, so this should also be backwards compatible in the future with such as change?
RalfJung
Apr 6, 2018
•
Member
The question was more: should the surface syntax be a thing?
by analogy with const fn(T) -> R.
Well, very much unlike const
, async
can be just syntactic sugar.
The question was more: should the surface syntax be a thing?
by analogy with const fn(T) -> R.
Well, very much unlike const
, async
can be just syntactic sugar.
I really love how this turned out (slightly disappointed that Surprisingly after how pro-"actual return type" I was I don't really mind how this looks. I think a big part of my revulsion with the current implementation is that the specific transform Despite liking the proposed return type treatment I still feel like the lack of being able to use named existential types is a downside; it removes the ability to further constrain a trait Client {
type Request<'a>: Future<Item = String> + 'a;
fn get(&mut self) -> Self::Request<'_>;
}
fn foo<C>(client: C) where C: Client, C::Request: Send {
...
} |
@Nemo157 The framing I described above, treating |
This comment has been hidden.
This comment has been hidden.
I'm not sure I personally prefer yet another keyword that can be put in front of fn, over an attribute, but I guess opinions on that differ. Anyway, while it is a keyword, would it make sense to explicitly state its position in relation to other such keywords? For example, |
One bikeshed alongside the idea of This would make it more clear that you're not running any of the function body until you actually
Making call syntax available in In fact, tokio::run(async {
let a = async {
... await!(foo(1, 2, 3)) ...
};
let b = async {
... await!(bar("a", "b", "c")) ...
};
await!(join(a, b))
}); This also has two future-proofing effects:
And an even smaller bikeshed: the list of things that |
@rpjohnst IMO the ergonomic difference between tokio::run(async {
let a = async {
... await!(foo(1, 2, 3)) ...
};
let b = async {
... await!(bar("a", "b", "c")) ...
};
await!(join!(a, b))
}); and tokio::run(async {
await!(join!(foo(1, 2, 3), bar("a", "b", "c")))
}); isn't worth whatever clarity benefits you gain from "forcing" |
Erm, I intended But you're right, it is a bit worse in that sense. On the other hand, it leaves open the option of implicit await, so it might look like this instead: tokio::run(async {
join(async { foo(1, 2, 3) }, async { bar("a", "b", "c") })
}); Basically, annotate the uncommon case of not immediately awaiting, and leave the common case of immediate awaiting noise-free. |
That's a cool idea, but again it creates a distinction between Explicit |
If |
@cramertj Ah, that's a good point. Depending on how common we expect On the other hand, I'm not sure that's any different from just calling a sync function. There's always the possibility that you'll need to look up a function signature, and the fact that So while we don't need to decide the particular await syntax yet, it would still be nice to be forward compatible with implicit await (i.e. not directly expose "
|
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
3. Define the precedence as the inconvient precedence - this seems equally | ||
surprising as the other precedence. | ||
4. Introduce a special syntax to handle the multiple applications, such as | ||
`await? future` - this seems very unusual in its own way. |
scottmcm
Apr 7, 2018
Member
- make
await
a postfix operator instead.
Every time I need (await Foo()).Bar
in C# I'm sad and wish it were something postfix.
- make
await
a postfix operator instead.
Every time I need (await Foo()).Bar
in C# I'm sad and wish it were something postfix.
est31
Apr 7, 2018
•
Contributor
What about Bar(Foo() await)
? :p
What about Bar(Foo() await)
? :p
Centril
Apr 7, 2018
Contributor
Could you elaborate on why it should be postfix?
I think await Foo()
reads better for an English (and at least Scandinavian) reader since you are (a)waiting for Foo()
to complete.
Could you elaborate on why it should be postfix?
I think await Foo()
reads better for an English (and at least Scandinavian) reader since you are (a)waiting for Foo()
to complete.
rpjohnst
Apr 7, 2018
It solves the precedence confusion around ?
and makes chaining easier. Basically the same arguments for going from try!
to ?
.
It solves the precedence confusion around ?
and makes chaining easier. Basically the same arguments for going from try!
to ?
.
scottmcm
Apr 7, 2018
•
Member
@Centril I agree it looks better for simple cases, but once you have multiple or need to mix it with methods and fields, it gets worse. I think x.foo().await.bar().await.qaz
(with highlighting) will look way better than (await (await x.foo()).bar()).qaz
-- fewer hard-to-track-down parens and maintains the reading order.
(Prefix would probably be better in Lisp or Haskell, but postfix seems generally better in rust. I sometimes even wish we had no prefix operators, so there'd never be a !(foo?)
vs (!foo)?
confusion...)
Edit: to show the difference keyword highlighting would make for emphasizing await points:
x.foo().let.bar().qaz.yay().let.blah
@Centril I agree it looks better for simple cases, but once you have multiple or need to mix it with methods and fields, it gets worse. I think x.foo().await.bar().await.qaz
(with highlighting) will look way better than (await (await x.foo()).bar()).qaz
-- fewer hard-to-track-down parens and maintains the reading order.
(Prefix would probably be better in Lisp or Haskell, but postfix seems generally better in rust. I sometimes even wish we had no prefix operators, so there'd never be a !(foo?)
vs (!foo)?
confusion...)
Edit: to show the difference keyword highlighting would make for emphasizing await points:
x.foo().let.bar().qaz.yay().let.blah
vi
Apr 8, 2018
•
In any case future await ?
should be added to the list as 5.
, without much analyzing if it is good or bad.
In any case future await ?
should be added to the list as 5.
, without much analyzing if it is good or bad.
vi
Apr 8, 2018
•
Like with the ?
instead of try!
, it puts the actual thing (what we are calling?) at the first place, and boilerplate-y technicalities (yield point or regular call?) at the end.
Like with the ?
instead of try!
, it puts the actual thing (what we are calling?) at the first place, and boilerplate-y technicalities (yield point or regular call?) at the end.
tmccombs
Apr 8, 2018
I like the idea of a postfix await operator, but I'm not sure what the best syntax for that would be.
f.await
looks too much like accessing a field f.await()
looks too much like a method call f await
is just as awkward to use as await f
.
I like the idea of a postfix await operator, but I'm not sure what the best syntax for that would be.
f.await
looks too much like accessing a field f.await()
looks too much like a method call f await
is just as awkward to use as await f
.
vi
Apr 9, 2018
Or it can be just some another special character, like future@?
or future~?
.
Or it can be just some another special character, like future@?
or future~?
.
@rpjohnst Would your concern about the syntax similarity between async and non-async fns leading to dropped futures be addressed by marking futures and streams as #[must_use], as is being discussed in the companion RFC? |
SGTM. I vaguely prefer @rpjohnst's implicit-
Calling a blocking function in async code would have already been a bug, wouldn't it? |
loop { | ||
match Future::poll(Pin::borrow(&mut pin), &mut ctx) { | ||
Async::Ready(item) => break item, | ||
Async::Pending => yield, |
Nemo157
Apr 7, 2018
Member
As @glaebhoerl mentioned on the companion RFC the Async
alias is not in the proposed API, these should be Poll::
.
As @glaebhoerl mentioned on the companion RFC the Async
alias is not in the proposed API, these should be Poll::
.
This comment has been hidden.
This comment has been hidden.
@withoutboats @Centril Does |
Cross-referencing: #2442 would allow a postfix await on an expression ( |
I am curious what is the difference between async Closure and async block? |
@warlord500 Async closures are like async functions in the regard that you need to call them to get a future. So, consistency is a reason. An async block is an expression that evaluates immediately to a future. We're also not sure whether they should allow using the |
For an example of where you would want to use each, async blocks are useful for having some synchronous immediately executed code when constructing a future: fn get(url: impl Into<Url>) -> impl Future<Output = Response> {
let url = url.into();
async {
// do the request
}
} while I mostly see async closures as useful for things like stream adaptors (this is including a few hypothetical apis, not sure how they'll really look) async fn sum_counts(urls: impl Iterator<impl Into<Url>>) -> Result<u32, Error> {
let counts = await!(stream::iter(urls).parallel(4).map(async |url| {
let res = await!(get(url))?;
let value: SomeStruct = await!(serde_json::from_async_reader(res.body()))?;
Ok(value.count)
}).collect::<Result<Vec<u32>>>())?;
Ok(counts.sum())
} |
Some musing about the possibility of tail-call optimization with From what I guess, the compiler will not even try tail-call optimizing an The only scheme I can think of would be to wrap the result of the For trait-object-based handling, I think it's already possible to do this exclusively with a third-party crate, that would just have to provide the said wrapping future type. For enum-based handling, I more or less think it cannot possibly be done while remaining sane. It would first require the compiler to provide in some way the closure of all future types that can be called as tail-calls from a future, and then a crate to actually wrap it in a nice wrapping-future type. Overall, I think the potential gain (enum vs trait object dispatch) is not worth the added complexity. |
@Ekleog Just as some additional context, there was a suggestion from @rpjohnst to have a CPS-style transformation added to generators/futures which would allow futures to drive themselves which would lend itself well to TCO. https://internals.rust-lang.org/t/pre-rfc-cps-transform-for-generators/7120/23 It's hard to say whether that scheme would be "zero-cost", as the generators would be storing the pointers on the stack, but it would save cycles from having the CPU do extra polling and assist TCO. |
Re: syntax for Has anyone considered using I personally prefer spelled-out keywords ( |
@oleganza Yes, |
* `async` from rust-lang/rfcs#2394; * `existential` from rust-lang/rfcs#2071.
* Add new keywords * `async` from rust-lang/rfcs#2394; * `existential` from rust-lang/rfcs#2071. * Make `existential` a contextual keyword Thanks @dlrobertson who let me use his PR #284!
Rendered
Companion libs RFC
Edit: Updated rendered link to merged location