Read cached output line-by-line #7737
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This avoids loading potentially gigabytes of output into memory, which can cause OOMs.
|
(rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
Thanks! |
|
📌 Commit 5e64197 has been approved by |
bors
added
S-waiting-on-bors
bors
added a commit
that referenced
this pull request
Dec 22, 2019
Read cached output line-by-line This avoids loading potentially gigabytes of output into memory, which can cause OOMs. Fixes #7736. This does not add a test as I don't really want to generate gigabytes of output (that seems like a bad idea) -- and it's unclear how to test other than by causing OOM on (most) CI systems, and it's unlikely that we'll actually regress here.
|
☀️ Test successful - checks-azure |
bors
added a commit
that referenced
this pull request
Jan 22, 2020
Fix cache replay including extra newlines. The compiler output cache replay was changed in #7737 to use `BufReader::read_line` instead of `str::lines`. `read_line`, unlike `lines`, includes the trailing line ending. The code is written assuming that the line endings are stripped, so make sure they are stripped here, too. This only happens for non-JSON messages, like `RUSTC_LOG`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This avoids loading potentially gigabytes of output into memory, which
can cause OOMs.
Fixes #7736.
This does not add a test as I don't really want to generate gigabytes of output (that seems like a bad idea) -- and it's unclear how to test other than by causing OOM on (most) CI systems, and it's unlikely that we'll actually regress here.