Further optimize off-heap traversal during minor GC #5195
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PB_ACTIVE_WRITER was set without PB_IS_WRITABLE
Only traverse first part of off-heap list that reside on the new-heap, except for writable binaries that are now kept in a separate list. Earlier during minor GC, traversing the off-heap list was continued to the end over terms residing on the old-heap. This for two reasons: 1. The BIN_OLD_VHEAP value was calculated by adding the sizes of all ProcBin's on the old-heap. 2. All writable binaries are handled by shrinking (reallocating) binaries that have not been written to since last GC. Even binaries residing on the old-heap. To optimize and only traverse the first part the off-heap list that reside on the new-heap, we now 1. maintain a correct value of BIN_OLD_VHEAP so it does not have to be recalculated during GC. 2. keep all writable binaries in a separate list Process.wrt_bins that is traversed to the end during every GC. The aim of this commit has been to maintain the overall GC semantics and especially not alter the value of BIN_OLD_VHEAP or how binaries have been reallocated at the end of a GC.
sverker
force-pushed
the
sverker/erts/writable-bin-list
branch
from
88f15eb
to
e16e545
Compare
garazdawi
added a commit
to garazdawi/otp
that referenced
this pull request
Feb 7, 2022
When creating a new proc bin because tracing has removed the writable flag from the proc bin, we must make sure to also create a new sub binary if the sub binary is on the mature or old heap. When this is not done, the subbinary can be promoted to the old heap without the proc bin also being there. Bug was introduced in erlang#5195, aka e16e545
garazdawi
added a commit
to garazdawi/otp
that referenced
this pull request
Feb 7, 2022
When creating a new proc bin because tracing has removed the writable flag from the proc bin, we must make sure to also create a new sub binary if the sub binary is on the mature or old heap. When this is not done, the subbinary can be promoted to the old heap without the proc bin also being there. Bug was introduced in erlang#5195, aka e16e545
garazdawi
added a commit
to garazdawi/otp
that referenced
this pull request
Feb 11, 2022
When creating a new proc bin because tracing has removed the writable flag from the proc bin, we must make sure to also create a new sub binary if the sub binary is on the mature or old heap. When this is not done, the subbinary can be promoted to the old heap without the proc bin also being there. Bug was introduced in erlang#5195, aka e16e545
max-au
added a commit
to max-au/otp
that referenced
this pull request
Dec 18, 2022
When writable binaries were introduced in erlang#5195, they weren't added to the output of `process_info(Pid, binary)`. It led to interesting effects of binaries suddenly disappearing after running implicit GC caused by BIF calls, including `process_info` itself. Effectively, running `process_info(self(), binary)` twice in a row resulted in different lists, because first call triggers GC clearing out original binary from the off-heap list: start() -> OffHeapBin = crypto:strong_rand_bytes(1024 * 1024), Bin = <<OffHeapBin/binary, "a">>, Before = erlang:process_info(self(), [binary]), After = erlang:process_info(self(), [binary]), Before =/= After andalso erlang:display({wtf, Before, After}), Bin. Fixes erlang#6573
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Only traverse first part of off-heap list that reside on the new-heap, except for writable binaries that are now kept in a separate list.
Fixes #4424.
Earlier during minor GC, traversing the off-heap list was continued to the end over terms residing on the old-heap. This for two reasons:
ProcBin's on the old-heap.To optimize and only traverse the first part the off-heap list that reside on the new-heap, we now
Process.wrt_binsthat is traversed to the end during every GC.The aim of this PR has been to maintain the overall GC semantics and especially not alter the value of BIN_OLD_VHEAP or how binaries have been reallocated at the end of a GC.