Skip to content

Check batchSizePerServer in Rebalance Config Pre-check#15984

Merged
somandal merged 5 commits intoapache:masterfrom
J-HowHuang:pre-check-batchsize-hotfix
Jun 4, 2025
Merged

Check batchSizePerServer in Rebalance Config Pre-check#15984
somandal merged 5 commits intoapache:masterfrom
J-HowHuang:pre-check-batchsize-hotfix

Conversation

@J-HowHuang
Copy link
Collaborator

Description

For the newly added rebalance config batchSizePerServer (#15617), this PR adds a new logic in rebalance config pre-check in table rebalance pre-checker to warn user when this parameter is recommended to be set.

When the number of segments to add to a single server is too large, batchSizePerServer is expected to be set to protect the server from stranded by helix messages. Currently this pre-check warns when any server will have more than 200 segments to add to, showing message to recommend user to set batchSizePerServer to 200 or lower.

Examples

Rebalance a table with batchSizePerServer=-1
image

Rebalance a table with batchSizePerServer=500
image

The same table with batchSizePerServer=200
image

@codecov-commenter
Copy link

codecov-commenter commented Jun 4, 2025

Codecov Report

Attention: Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.

Project coverage is 63.42%. Comparing base (1a476de) to head (8854e27).
Report is 194 commits behind head on master.

Files with missing lines Patch % Lines
...lix/core/rebalance/DefaultRebalancePreChecker.java 87.50% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #15984      +/-   ##
============================================
+ Coverage     62.90%   63.42%   +0.52%     
+ Complexity     1386     1355      -31     
============================================
  Files          2867     2910      +43     
  Lines        163354   166846    +3492     
  Branches      24952    25518     +566     
============================================
+ Hits         102755   105826    +3071     
- Misses        52847    53025     +178     
- Partials       7752     7995     +243     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.39% <92.30%> (+0.52%) ⬆️
java-21 63.36% <92.30%> (+0.54%) ⬆️
skip-bytebuffers-false ?
skip-bytebuffers-true ?
temurin 63.42% <92.30%> (+0.52%) ⬆️
unittests 63.42% <92.30%> (+0.52%) ⬆️
unittests1 56.51% <ø> (+0.69%) ⬆️
unittests2 33.47% <92.30%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@somandal somandal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@somandal somandal merged commit 6dccd39 into apache:master Jun 4, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants