Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement for_each_concurrent() #57

Closed
wants to merge 7 commits into from
Closed

Implement for_each_concurrent() #57

wants to merge 7 commits into from

Conversation

notgull
Copy link
Member

@notgull notgull commented Aug 28, 2022

This PR adds a new type, UnorderedFutures, that stores and then polls a series of futures concurrently. I then go on to use UnorderedFutures to implement for_each_concurrent() and try_for_each_concurrent, resolving #26.

The implementation I use here is relatively inefficient. It uses two allocations per future stored; one for the Container, and another for the readiness flag. In addition, another allocation is added per poll for the waker_fn. Ideally, it would look more like this:

pub struct UnorderedFutures<Fut> {
    first: Option<Pin<Arc<Container<Fut>>>>,
}

pin_project! {
    struct Container<Fut> {
        #[pin]
        future: Mutex<Fut>,
        next: Mutex<Option<Pin<Arc<Container<Fut>>>>>,
        ready: AtomicBool,
        waker: AtomicWaker,
    }
}

impl<Fut> Wake for Container<Fut> {
    // set the ready flag to true and then wake the waker
}

Note that this would add a dependency on libstd and atomic-waker, and would require unsafe code since it's impossible to get a Pin<MutexGuard<T>> from a Pin<&Mutex<T>> without unsafe code.

However, since this is an implementation detail that can be resolved through a patch bump in the future, I decided to leave it with the as-is simple implementation.

@taiki-e
Copy link
Collaborator

taiki-e commented Sep 5, 2022

UnorderedFutures seems to be a simpler version of futures's FuturesUnordered, but it's not clear to me how reasonable this is for this library, given the footgun in FuturesUnordered.

Also, as we cover the many APIs that futures has, eventually the benefits of this library will be lost...

@notgull
Copy link
Member Author

notgull commented Sep 5, 2022

You may be right; I wasn't aware of the footgun until now. In that case, I'll close this PR; however, we might want to explicitly document the best ways to concurrently run futures.

@notgull notgull closed this Sep 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants