improvement: allow for more concurrency when using cloud builder #5955
+198
−20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stefreak
requested review from
vvagaytsev and
twelvemo
and removed request for
vvagaytsev
- Allow more fine-grained status and execute task node concurrency control in the plugins using setters on the action. - Assign concurrency group by node execution type (status, execute), action type (exec, container) and the concurrency setting (assuming that actions with a separate max concurrency setting are a different workload class). - Fine-tune the build concurrency settings in the container and kubernetes plugins in the `validate` and `getStatus` handlers for every action. - This speeds up `garden build -f` in an internal project from ~40 seconds to ~14 seconds.
stefreak
force-pushed
the
cloudbuilder-concurrency
branch
from
42cfc0a
to
aec0082
Compare
vvagaytsev
reviewed
Apr 18, 2024
vvagaytsev
reviewed
Apr 18, 2024
Comment on lines
+35
to
+36
| override defaultStatusConcurrencyLimit = 10 | ||
| override defaultExecuteConcurrencyLimit = 10 |
There was a problem hiding this comment.
It looks like these overrides are not necessary because the same limits are already defined in the ExecuteActionTask super class.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
plugins using setters on the action.
(exec, container) and the concurrency setting (assuming that actions with a separate
max concurrency setting are a different workload class).
validateandgetStatushandlers for every action.garden build -fin an internal project from ~40 seconds to ~14 seconds.