Fix remote cache writes memory exhaustion. (#12087) #12089
Conversation
This builds upon pantsbuild#12083 which halved memory consumption when uploading blobs to the remote cache and sets an upper bound on memory consumed for uploads via two measures: + Blobs larger than a certain size are buffered via disk. + Uploading duplicate (large) blobs is largely avoided. The latter is achieved with a best-effort, low cost sieve. We could do better. With these changes, a full fresh remote cache upload of Pants via `./pants --remote-cache-write tests ::` can now complete (previously it was OOMKilled) and does so while consuming at least 30x less memory. (cherry picked from commit 68921c2) # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]
|
In case this is deemed safe enough for 2.5.1rc0, putting up the cherry-pick now. |
I assume this PR only affects remote cache writes? If so, then it has my vote for cherry-pick since having a large percentage of runs killed for OOM is pretty major bug for that feature. |
|
Is this the cherry-pick PR? The PR says it it to be merged into |
|
I always cherry pick via |
|
I believe it is more of a Pants project style question. cc @Eric-Arellano |
|
Yeah. Eric has been accepting this style in reviews for awhile. From me at least. |
|
Yeah, we simply update the release's changelog PR title to do the "right thing". |
This builds upon #12083 which halved memory consumption when uploading
blobs to the remote cache and sets an upper bound on memory consumed for
uploads via two measures:
The latter is achieved with a best-effort, low cost sieve. We could do
better.
With these changes, a full fresh remote cache upload of Pants via
./pants --remote-cache-write tests ::can now complete (previously itwas OOMKilled) and does so while consuming at least 30x less memory.
(cherry picked from commit 68921c2)
[ci skip-build-wheels]