Scheduler: Support parallel-install with merge-wait #1287
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zmedico
force-pushed
the
bug_925213_merge-wait_and_parallel-install
branch
8 times, most recently
from
a5762e8
to
c474cbe
Compare
zmedico
commented
Feb 26, 2024
thesamesam
approved these changes
Feb 27, 2024
thesamesam
reviewed
Feb 27, 2024
man/make.conf.5
Outdated
| disable \fIebuild\-locks\fR. | ||
| parallelism. For additional parallelism disable \fIebuild\-locks\fR. | ||
| Also disable \fImerge\-wait\fR for additional parallelism if desired, | ||
| but this is not necessary because \fIparallel\-install\fR cooperates |
There was a problem hiding this comment.
Maybe tweak the phrasing here? I think users get confused by the different options and how to achieve maximum parallelism (and the risks of each).
There was a problem hiding this comment.
Yeah, changed it to:
Also disable merge-wait for additional parallelism if desired, but that increases the possibility of random build failures. When parallel-install is used together with merge-wait, parallel installation occurs in batches when there are no builds running.
For system packages, always serialize install regardless of parallel-install, in order to mitigate failures triggered by fragile states as in bug 256616. For other packages, continue to populate self._task_queues.merge, which will serialize install unless parallel-install is enabled. Fixes: 825db01 ("Add merge-wait FEATURES setting enabled by default") Bug: https://bugs.gentoo.org/256616 Bug: https://bugs.gentoo.org/925213 Signed-off-by: Zac Medico <zmedico@gentoo.org>
zmedico
force-pushed
the
bug_925213_merge-wait_and_parallel-install
branch
from
c474cbe
to
0896ede
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For system packages, always serialize install regardless parallel-install, in order to mitigate failures triggered by fragile states as in bug 256616. For other packages, continue to populate self._task_queues.merge, which will serialize install unless parallel-install is enabled.
Fixes: 825db01 ("Add merge-wait FEATURES setting enabled by default")
Bug: https://bugs.gentoo.org/256616
Bug: https://bugs.gentoo.org/925213