HBASE-24659 Calculate FIXED_OVERHEAD automatically #2018
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Apache9
requested changes
Jul 5, 2020
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
saintstack
reviewed
Jul 6, 2020
| 3 * Bytes.SIZEOF_INT + | ||
| 14 * Bytes.SIZEOF_LONG + | ||
| 3 * Bytes.SIZEOF_BOOLEAN); | ||
| public static final long FIXED_OVERHEAD = ClassSize.estimateBase(HRegion.class, false); |
There was a problem hiding this comment.
Does ClassSize come up w/ same general numbers as old manual technique. It does deep size rather than shallow?
There was a problem hiding this comment.
I think ClassSize.estimateBase only calculate shallow size
There was a problem hiding this comment.
I thought it critical it did deep size? Do you get the same numbers roughly?
Yes, this change only affect class loading phase. And the calculation of FIXED_OVERHEAD cost less than 5 milliseconds |
|
🎊 +1 overall
This message was automatically generated. |
|
Let's run a YCSB or at least a PE to see the performance impact? |
yes,it is necessary, Let me do it |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
@Apache9 sir,I have done the performance test, and attaches on jira page, please take a look ? |
Apache9
approved these changes
Aug 1, 2020
|
@saintstack PTAL. If no objections, I will push this to master and branch-2. Thanks. |
|
@saintstack please have a look, thanks |
|
Oh, forgot to merge this one. Since you have pinged @saintstack again, let's wait until tomorrow... |
|
Pardon me for missing your pings @nyl3532016 ... let me look. |
saintstack
approved these changes
Aug 6, 2020
There was a problem hiding this comment.
Thank you for waiting on my input. I had concerns but after checking this patch again and going back through git history, I see better what is going on and +1 this approach. Thank you for running the PE check. Good. This PR might make our class loading take a little longer but should have no effect on general perf I'd think. Thanks for the patch.
| 4 * Bytes.SIZEOF_BOOLEAN + Bytes.SIZEOF_LONG + | ||
| //byte[] headers for column family and table name | ||
| 2 * ClassSize.ARRAY + 2 * ClassSize.REFERENCE); | ||
| public static final long FIXED_OVERHEAD = ClassSize.estimateBase(HFileContext.class, false); |
There was a problem hiding this comment.
Or, sorry, yeah, FIXED_OVERHEAD is 'this' classes size; i.e. the shallow size (I think).
There was a problem hiding this comment.
I went back through history of ClassSize... its so old.
saintstack
pushed a commit
that referenced
this pull request
Aug 6, 2020
Co-authored-by: niuyulin <niuyulin@xiaomi.com> SIgned-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: stack <stack@apache.org>
clarax
pushed a commit
to clarax/hbase
that referenced
this pull request
Nov 15, 2020
Co-authored-by: niuyulin <niuyulin@xiaomi.com> SIgned-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: stack <stack@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.