Remote: Fix performance regression in "upload missing inputs". #15890
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I tested locally that the "collect digests" part is back to normal. @clint-stripe Can you please give it a try? |
|
@bazel-io flag |
bazel-io
added
the
potential release blocker
|
Awesome! This does indeed fix it for our 600k-input action as well; the trace in perfetto looks much better!
|
coeuvre
force-pushed
the
test-upload-inputs-head
branch
3 times, most recently
from
6c2b485
to
a2be64d
Compare
sgowroji
added
team-Remote-Exec
|
@bazel-io fork 5.3.0 |
bazel-io
removed
the
potential release blocker
|
@ckolli5 is this something we could cherry-pick into 5.2? We're stuck on Bazel 5.0 without this fix, and waiting until 5.3 is released in August is not ideal. |
|
@clint-stripe Making a 5.2.1 release to cherry-picking this change won't be much faster than the 5.3 release, we still have to go through the same release process. So please wait for 5.3, thanks! |
|
It might be helpful to get an RC out for 5.3 (or some bazelisk pointable binary). We have lots of changes in the branch, but no built binaries to test yet. |
coeuvre
force-pushed
the
test-upload-inputs-head
branch
2 times, most recently
from
a32df05
to
d5363a3
Compare
sgowroji
added
awaiting-user-response
|
@brentleyjones Actually, you can use |
coeuvre
force-pushed
the
test-upload-inputs-head
branch
from
d5363a3
to
b576aff
Compare
Also add more tests. Fixes bazelbuild#15872.
coeuvre
force-pushed
the
test-upload-inputs-head
branch
from
b576aff
to
3e08a8d
Compare
sgowroji
added
awaiting-review
coeuvre
added a commit
to coeuvre/bazel
that referenced
this pull request
Jul 28, 2022
The regression was introduced in 702df84 where we essentially create a subscriber for each digest to subscribe the result of `findMissingBlobs`. This change update the code to not create so many subscribers but maintain the same functionalities. Fixes bazelbuild#15872. Closes bazelbuild#15890. PiperOrigin-RevId: 463826260 Change-Id: Id0b1c7c309fc9653a47c5df95c609b34e6510cde
ShreeM01
pushed a commit
that referenced
this pull request
Jul 28, 2022
The regression was introduced in 702df84 where we essentially create a subscriber for each digest to subscribe the result of `findMissingBlobs`. This change update the code to not create so many subscribers but maintain the same functionalities. Fixes #15872. Closes #15890. PiperOrigin-RevId: 463826260 Change-Id: Id0b1c7c309fc9653a47c5df95c609b34e6510cde
ShreeM01
removed
the
awaiting-review
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The regression was introduced in 702df84 where we essentially create a subscriber for each digest to subscribe the result of
findMissingBlobs.This change update the code to not create so many subscribers but maintain the same functionalities.
Fixes #15872.