Skip to content

Comments

Refactor WagedRebalancer and add comments#2431

Merged
xyuanlu merged 2 commits intoapache:masterfrom
qqu0127:waged-refactor
Apr 6, 2023
Merged

Refactor WagedRebalancer and add comments#2431
xyuanlu merged 2 commits intoapache:masterfrom
qqu0127:waged-refactor

Conversation

@qqu0127
Copy link
Contributor

@qqu0127 qqu0127 commented Apr 3, 2023

Create standalone classes for partial and global rebalance to make the class more modular and easier to manage.

Issues

  • My PR addresses the following Helix issues and references them in the PR description:

Resolves #2430

Description

  • Here are some details about my PR, including screenshots of any UI changes:

Refactor WagedRebalancer by separating out components to their standalone classes.
Currently the WagedRebalancer class is complicated and contains multiple processes. With everything in one class, it's hard to read and even worse to manage complexity.
Creating their classes for partial and global rebanace have a clear cut on responsibility, and make it possible for future refactoring toward microservices.

This PR does the following:

  1. Move partial and global rebalance process from WagedRebalancer to their own classes
  2. Remove process-specific member variables and methods from WagedRebalancer
  3. Add clarifying comments for partial and global rebalance.

No new logic is introduced, this PR is pure refactoring.

Tests

  • The following tests are written for this issue:

Existing tests in TestWagedRebalancer

  • The following is the result of the "mvn test" command on the appropriate module:

(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)

Test for TestWagedRebalancer
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.535 s - in org.apache.helix.controller.rebalancer.waged.TestWagedRebalancer
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.6:report (generate-code-coverage-report) @ helix-core ---
[INFO] Loading execution data file /Users/qqu/workspace/qqu-helix/helix-core/target/jacoco.exec
[INFO] Analyzed bundle 'Apache Helix :: Core' with 936 classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 40.842 s
[INFO] Finished at: 2023-04-03T15:42:46-04:00
[INFO] ------------------------------------------------------------------------

Changes that Break Backward Compatibility (Optional)

  • My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:

(Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)

Documentation (Optional)

  • In case of new functionality, my PR adds documentation in the following wiki page:

(Link the GitHub wiki you added)

Commits

  • My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Code Quality

  • My diff has been formatted using helix-style.xml
    (helix-style-intellij.xml if IntelliJ IDE is used)

Create standalone classes for partial and global rebalance to make the
class more modular and easier to manage.
Copy link
Contributor

@junkaixue junkaixue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Good job!

Copy link
Contributor

@desaikomal desaikomal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is extremely good change. Splitting the functionality is very useful.
It is not possible to look through each and every line of change as the change is really big for human brain to perform.
Hopefully lift-n-shift worked. AFAIK you didn't fundamentally changed any behavior. just code alignments.
Amazingly cool

@qqu0127
Copy link
Contributor Author

qqu0127 commented Apr 4, 2023

This PR is ready to merge, approved by @junkaixue
Please use the original commit message:

Refactor WagedRebalancer and add comments
Create standalone classes for partial and global rebalance to make the class more modular and easier to manage.

Copy link
Contributor

@rahulrane50 rahulrane50 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very clean! Good job Quincy! I just wish github has"moved code block" feature. it would have been very easy to make sure we don't miss anything in refactoring :)

* assignmentMetadataStore, return the current state assignment.
* @throws HelixRebalanceException
*/
public Map<String, ResourceAssignment> getBestPossibleAssignment(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. Actually these two methods are almost identical except "getBaseline()" vs "getBestPossibleAssignment()" call. I totally understand if we want keep both of them but just wondering if changing signature of this method (AssignmentMetadataStore --> func) might help to combine both of this methods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review.
That's a good idea should be doable, but let's leave that for future refactoring as this PR is big enough.

@xyuanlu
Copy link
Contributor

xyuanlu commented Apr 6, 2023

Thanks for making the effort on code clean and refactor!

@xyuanlu xyuanlu merged commit 0e93568 into apache:master Apr 6, 2023
rahulrane50 pushed a commit to rahulrane50/helix that referenced this pull request May 31, 2023
Refactor WagedRebalancer and add comments

Create standalone classes for partial and global rebalance to make the
class more modular and easier to manage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WagedRebalancer should be better modularized

5 participants