-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Limit concurrency of asset copy tasks #11708
Conversation
This would deserve a description with why and how it's done. Other than that... |
Why: parallelism after a certain point doesn't make things any faster: there's only so much bandwidth. Less parallelism increases the amount of feedback on progress and allows retries of interrupted copies to better avoid recopying by having more completed copies. How: It uses a buffered channel as a semaphore. |
@hakman Is there a particular place you wanted that description? How can I move this forward? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, missed this.
Description was not for me but for anyone looking at this 6 months from now.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hakman The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Why: parallelism after a certain point doesn't make things any faster: there's only so much bandwidth. Less parallelism increases the amount of feedback on progress and allows retries of interrupted copies to better avoid recopying by having more completed copies.
How: It uses a buffered channel as a semaphore.