HBASE-26147: Add a dry run mode to the balancer, where moves are calculated but not actually executed #3630
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bbeaudreault
force-pushed
the
dry_run_balancer_master
branch
from
b00398b
to
df64523
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
The HBaseTestingUtility has been renamed to HBaseTestingUtil on master, which cause the compilation error. Anyway, I think this means we should try to push out 3.0.0 ASAP, as there are more and more incompatible changes between master and branch-2. |
|
Thank you Duo. I didn't notice this because my maven threw a NPE locally rather than any useful compile error (this jvm bug), so I was hoping the build server would help me. I ended up figuring it out and fixing the HBaseTestingUtil conflict by setting I agree it'd be nice to get moving on 3.0.0, it was pretty painful to port this particular patch. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
…ulated but not actually executed
bbeaudreault
force-pushed
the
dry_run_balancer_master
branch
from
df64523
to
4000c72
Compare
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Apache9
reviewed
Aug 27, 2021
| * Encapsulates options for executing a run of the Balancer. | ||
| */ | ||
| @InterfaceAudience.Public | ||
| @InterfaceStability.Evolving |
There was a problem hiding this comment.
There is no IS.Evolving for IA.Public classes, it should always be IS.Stable implicitly, so we do not add IS annotation for IA.Public classes.
There was a problem hiding this comment.
Sorry, I misunderstood. I will remove. I have to say that it's a little unfortunate that there's no way with the current compatibility matrix to mark something as effectively beta. That's what I thought Evolving was for, and part of the reason we decided to break this out of the more stable Admin interface.
There was a problem hiding this comment.
@Apache9 I'm also surprised by your interpretation of the interaction between IA.Public and IS.Evolving. The javadoc on InterfaceStability says
All classes that are annotated with {@link Public} or {@link LimitedPrivate} must have InterfaceStability annotation.
Indeed, InterfaceStability itself is annotated with IA.Public and IS.Evolving.
| * Builder for constructing a {@link BalanceRequest} | ||
| */ | ||
| @InterfaceAudience.Public | ||
| @InterfaceStability.Evolving |
| * Response returned from a balancer invocation | ||
| */ | ||
| @InterfaceAudience.Public | ||
| @InterfaceStability.Evolving |
| * Builds a {@link BalanceResponse} for returning results of a balance invocation to callers | ||
| */ | ||
| @InterfaceAudience.Public | ||
| @InterfaceStability.Evolving |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Apache9
approved these changes
Aug 28, 2021
joshelser
approved these changes
Aug 30, 2021
There was a problem hiding this comment.
LGTM. Sorry I forgot that Public+Evolving was a no-go (you're right that we should have something in-between -- LimitedPrivate+Evolving might be the way to go but it's hard when we're talking about Admin and other long-standing API).
Only one final thought is that BalanceResponse$Builder probably shouldn't be Public. Use of the object is certainly Public, but how that object is created does not need to be public.
I think I'd rather see this land rather than another round of reviews. The last precommit round looks good to me (just some flakiness in tests). Let me try to do that tonight.
| /** | ||
| * Builds a {@link BalanceResponse} for returning results of a balance invocation to callers | ||
| */ | ||
| @InterfaceAudience.Public |
There was a problem hiding this comment.
Thinking about it, we probably don't want the Builder class here to be Public. Users shouldn't ever be building this BalanceResponse on their own, just using the BalanceResponse.
There was a problem hiding this comment.
I created https://issues.apache.org/jira/browse/HBASE-26240 for this. I'll submit a small PR once these existing PRs are merged.
|
I left a comment on one of Duo's line comments, but I'll repeat it here. I'm also surprised by this interpretation of the interaction between
Indeed, @joshelser @Apache9 do we document the HBase project's interpretation of these annotations somewhere in our developer guide? |
Yeah, we have this: https://hbase.apache.org/book.html#hbase.client.api.surface I think Bryan definitely found a gap in that we don't have a good way to represent user-facing but not yet stable. |
|
Best as I can tell, the test failures were flakes which passed on re-run. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the master port for #3536
I also addressed documentation feedback from #3536. This port was not super straightforward because of rsgroup being moved into
hbase-server. The move was not a simply file move, but code moved within various classes. Additionally, HBaseAdmin.java was removed in master in favor of AdminOverAsyncAdmin (and some other minor admin changes).I've read everything over, but it may be worth giving a quick look for correctness