Merged
Conversation
Rewrite MemoryConsumingIterator's method to compute the amount of memory to consume, so that: * The implementation is more comprehensible * Replace exception with bounds checking * Avoid allocating more than necessary (a single byte is sufficient) * The log message includes the amount of used memory detected * The waiting message appears, even when memory was detected, because that's the behavior * Give the GC more time to detect the changed GC condition before trying to detect the low memory condition Also, remove hard-coded comments for size of heap and incorrect interval frequency, and increase the configured free memory threshold, so that the memory percentage isn't so low, it doesn't get lower than the minimum that G1GC needs to do its job by default on a 256K VM. This fixes apache#3868 Also include trivial fixes: * Fix deprecation warning issues for getSplitCreationTime by making impl class deprecated instead of suppressing the interface deprecation, and use regular deprecations, not forRemoval=true, which complicates the way deprecations get inherited (see comments on apache#3977) * Remove unused Logger
Member
Author
|
These changes finished a build with the full ITs, and these tests did not fail... they didn't even flake. They seem to pass consistently now with these changes. |
Contributor
|
As I mentioned here, I think the deprecation changes should be removed from this PR and included in a new PR that also includes the |
dlmarion
approved these changes
Dec 6, 2023
Contributor
dlmarion
left a comment
dlmarion
left a comment
There was a problem hiding this comment.
The MemoryStarved*IT changes look good. I have a suggestion regarding the deprecation changes elsewhere.
ctubbsii
commented
Dec 6, 2023
Member
Author
ctubbsii
left a comment
ctubbsii
left a comment
There was a problem hiding this comment.
Restore unrelated forRemoval=true stuff.
core/src/main/java/org/apache/accumulo/core/spi/balancer/SimpleLoadBalancer.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/apache/accumulo/core/spi/balancer/data/TabletStatistics.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/apache/accumulo/core/manager/balancer/TabletStatisticsImpl.java
Outdated
Show resolved
Hide resolved
Member
Author
|
I created #4032 for the forRemoval stuff. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrite MemoryConsumingIterator's method to compute the amount of memory to consume, so that:
Also, remove hard-coded comments for size of heap and incorrect interval frequency, and increase the configured free memory threshold, so that the memory percentage isn't so low, it doesn't get lower than the minimum that G1GC needs to do its job by default on a 256K VM.
This fixes #3868
Also include trivial fixes: