Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upProvide `select!` macro #2152
Provide `select!` macro #2152
Conversation
Provides a `select!` macro for concurrently waiting on multiple async expressions. The macro has similar goals and syntax as the one provided by the `futures` crate, but differs significantly in implementation. First, this implementation does not require special traits to be implemented on futures or streams (i.e., no `FuseFuture`). A design goal is to be able to pass a "plain" async fn result into the select! macro. Even without `FuseFuture`, this `select!` implementation is able to handle all cases the `futures::select!` macro can handle. It does this by supporting pre-poll conditions on branches and result pattern matching. For pre-conditions, each branch is able to include a condition that disables the branch if it evaluates to false. This allows the user to guard futures that have already been polled, preventing double polling. Pattern matching can be used to disable streams that complete. A second big difference is the macro is implemented almost entirely as a declarative macro. The biggest advantage to using this strategy is that the user will not need to alter the rustc recursion limit except in the most extreme cases. The resulting future also tends to be smaller in many cases.
This comment has been minimized.
This comment has been minimized.
|
Of course, I should add that none of this would be possible w/o @cramertj doing the bulk of the initial exploration via |
|
LGTM, only minor nits. Well done! |
|
This is awesome! Great work! |
This makes the borrow checker happy
This comment has been minimized.
This comment has been minimized.
|
This looks neat! I'm glad to have a second set of eyes on this. futures-rs also initially used a declarative macro approach but I moved away from it since I had trouble getting it to give good error messages, and the code was harder to maintain. The way you wrote this looks much cleaner! I'm curious about the I'm actually curious about your examples in this regard. I don't understand how this loop wouldn't poll while rem {
tokio::select! {
Some(x) = rx1.recv() => {
msgs.push(x);
}
Some(y) = rx2.recv() => {
msgs.push(y);
}
else => {
rem = false;
}
}
}The first time |
This comment has been minimized.
This comment has been minimized.
|
@cramertj Thanks for the kind words.
I expect the declarative macro will give wonky error messages in some cases. This is one reason why I left a proc macro (that generates the
It does poll |
This comment has been minimized.
This comment has been minimized.
|
As for whether or not it is a foot gun, it seems no more of a foot gun than |
|
This is very cool! I'm personally not a huge fan of the syntax used by any select macro (the |
This comment has been minimized.
This comment has been minimized.
mystor
commented
Jan 23, 2020
|
I think it's possible to entirely avoid the The main change is using a |
This comment has been minimized.
This comment has been minimized.
|
@mystor yep! you are correct. I opted to keep a proc macro component in order to provide improved error messages when invalid syntax is used. |
This comment has been minimized.
This comment has been minimized.
mystor
commented
Jan 23, 2020
|
Sounds good :-) - I think some of the error messages produced by the proc-macro, such as "up to 64 branches supported", could be handled by a |
This comment has been minimized.
This comment has been minimized.
|
I expect forgetting a |
This comment has been minimized.
This comment has been minimized.
loyd
commented
Jan 28, 2020
|
Now it's incredibly easy to get panic using |
This comment has been minimized.
This comment has been minimized.
withoutboats
commented
Jan 28, 2020
This doesn't seem comparable to me:
To me it seems sort of the opposite: with iterator, we have made it very easy to avoid a low stakes bug, but with this select design the user is fully responsible ofr avoiding a high stakes bug. Personally, I wish we had just made I assume (from omission) that this doesn't change anything about the relationship between Unpin and select, so I'd just mention my own obsession with the select API: I think the vast majority of select use cases could be solved with two higher level APIs, both of which could pin futures themselves and not require unpin:
Just mentioning these in case you're interested in exploring other concurrency primitives that could be a bit higher level than select and easier to use. |
This comment has been minimized.
This comment has been minimized.
|
@withoutboats thanks for the thoughts. Based on your description of |
This comment has been minimized.
This comment has been minimized.
|
@loyd it is not "incredibly easy", the future needs to be both |
carllerche commentedJan 22, 2020
•
edited by udoprog
The
select!operation is a key operation when writing asynchronous Rust. Up until now, thefuturescrate provided the main implementation. This PR adds a newselect!implementation that improves on the version fromfuturesin a few ways:FusedFuture.proc-macro-hack.Avoiding
FusedFutureThe original
select!macro requires that provided futures implement a special trait:FusedFuture. Unfortunately, futures returned byasync fndo not implementFusedFuture, leading to the requirement of having to call.fuse()on futures before callingselect!. TheFuseFuturerequirement exists to support usingselect!from within a loop.The
select!implementation in this PR avoids the need for aFusedFuturetrait by adding two new features toselect!: pattern matching and branch conditions.The core of the "
select!in a loop" problem is that future values may not be used again after they complete, so when usingselect!from within a loop, branches that have completed on prior loop iterations must somehow be "disabled". The strategy used byfutures::select!is to require input futures to implement [FusedFuture]. TheFusedFuturetrait informsselect!if the branch is to be disabled.In this PR, we use branch conditions. This idea was initially proposed here by @Matthias247. The idea is that the user may supply a condition to guard a branch and informing
select!to disable the branch if the future has previously completed. Here is an example of implementingjoinwithselect!.Additionally, this PR builds upon the condition idea by also adding pattern matching. This provides a "post condition" ability where a select branch can be disabled after the future completes. Here is an example with selecting on streams:
The
elsebranch is executed if all branches of aselectcall become disabled. This is equivalent to thecompletedbranch infutures::select!.Note that this macro does not support
defaultbranches. This behavior is orthogonal toselect!and can be implementing using a separate utility.Implementing (mostly) as a declarative macro
This
select!implementation is implemented mostly as a declarative macro. Doing so avoids the need for the user to increase the rustc recursion limit. The macro recurses once per branch, so over 60 select! branches can be used before having to touch the recursion limit.The decision to avoid a proc macro stems mostly from the fact that proc macros in expression position are not supported in stable Rust. The
proc-macro-hackcrate exists as a work around, but runs into limitations with regards to nesting.proc-macro-nestedsupports nestingproc-macro-hackcalls, but results in hitting the rustc recursion limit very early. This, of course, is not a criticism ofproc-macro-hackas @dtolnay does wonders with what is available. Theselect!implementation uses some tricks learned from readingproc-macro-hacksource.