Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add concurrency limit option to package lo/parallel #77

Open
Bin-Huang opened this issue Mar 25, 2022 · 1 comment
Open

Add concurrency limit option to package lo/parallel #77

Bin-Huang opened this issue Mar 25, 2022 · 1 comment

Comments

@Bin-Huang
Copy link
Contributor

Sometimes if the collection is too big, it will make too many goroutines in one time when using parallel methods. That will significantly increase resource usage, especially when the iteratee function is computationally-Intensive, io-Intensive, or both.

An example that requests every URL in a huge list, will start so many requests at the same time:

hugeUrlList := []string{ /*...*/ }
parallel.ForEach(hugeUrlList, request)

I think there should have an option to set the max concurrency count, which means up to configured number iteratee calling at the same time:

hugeUrlList := []string{ /*...*/ }
parallel.ForEach(hugeUrlList, request, parallel.Option().Concurrency(100))

I will enjoy implementing it.

@Azer0s
Copy link
Contributor

Azer0s commented Mar 27, 2022

This is already implemented in #37

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants