Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic StreamExt::{flatten_unordered, flat_map_unordered} impls #2083

Merged
merged 23 commits into from Feb 6, 2022

Conversation

olegnn
Copy link
Contributor

@olegnn olegnn commented Feb 17, 2020

Closes #1781
Closes #2378

Basic implementations of flatten_unordered and flat_map_unordered using FuturesUnordered in combination with custom poll state and wakers under the hood. I didn't measure performance but I think it's fast enough. It differs from flatten and flat_map because it polls stream and inner streams concurrently.

Edited: I added benchmarks, which can be compared with futures_unordered. flatten_unordered with a stream of 10000 oneshots each of which produces an inner stream with one item taking approx about x1.5 time of the original futures_unordered benchmark with 10000 oneshots.

New update: after changing the logic to make a stream being polled in a loop instead of once per poll_next, flatten_unordered takes about 1 - 1.1x time of the original futures_unordered benchmark.

@olegnn olegnn force-pushed the stream_flat_map_unordered branch from c91d08b to 090d95d Compare Feb 17, 2020
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from 4b94925 to dae2bb9 Compare Feb 17, 2020
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from 0b40d25 to bd9a86c Compare Feb 18, 2020
@olegnn olegnn changed the title Basic StreamExt::flat_map_unordered impl Basic StreamExt::{flatten_unordered, flat_map_unordered} impls May 17, 2020
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 2 times, most recently from 5ba808d to 6adcfee Compare May 17, 2020
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 2 times, most recently from 2ca3e96 to f2ddcfd Compare May 28, 2020
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 8 times, most recently from e402c5f to 1324ea9 Compare Jun 12, 2020
@olegnn olegnn requested a review from taiki-e Jan 9, 2022
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 3 times, most recently from fe55019 to 952ccf7 Compare Jan 9, 2022
Copy link
Member

@taiki-e taiki-e left a comment

Looks great!

futures-util/src/stream/stream/flatten_unordered.rs Outdated Show resolved Hide resolved
futures-util/src/stream/stream/mod.rs Outdated Show resolved Hide resolved
futures/tests/stream.rs Outdated Show resolved Hide resolved
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from bf4a826 to 12bee98 Compare Jan 18, 2022
futures/tests/stream.rs Outdated Show resolved Hide resolved
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from d632a0f to 4091d66 Compare Jan 23, 2022
@olegnn olegnn requested a review from taiki-e Jan 26, 2022
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 2 times, most recently from 6096ba6 to 330d93d Compare Jan 26, 2022
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 3 times, most recently from f55cdcd to c6a9357 Compare Jan 26, 2022
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from c6a9357 to 69f60cc Compare Jan 26, 2022
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from d600a6a to a50c9e4 Compare Jan 27, 2022
taiki-e
taiki-e approved these changes Feb 6, 2022
Copy link
Member

@taiki-e taiki-e left a comment

Thanks!

@taiki-e taiki-e merged commit 9cb6723 into rust-lang:master Feb 6, 2022
23 checks passed
@taiki-e taiki-e added the 0.3-backport: pending The maintainer accepted to backport this to the 0.3 branch, but backport has not been done yet. label Feb 6, 2022
@taiki-e taiki-e mentioned this pull request Feb 6, 2022
@taiki-e taiki-e added 0.3-backport: completed and removed 0.3-backport: pending The maintainer accepted to backport this to the 0.3 branch, but backport has not been done yet. labels Feb 6, 2022
@taiki-e
Copy link
Member

taiki-e commented Feb 6, 2022

Published in 0.3.20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants