New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core, schedule: use size as a factor of score #830

Merged
merged 9 commits into from Nov 8, 2017

Conversation

Projects
None yet
4 participants
@Connor1996
Member

Connor1996 commented Nov 2, 2017

When calculate region/leader score, use size instead of count, in order to balance store size in final.

new shouldBalance logic:

  1. introduce avgScore to make sure moving is not unnecessary. Cause in some case:
    note that: avgScore is not equal to totalScore / count, since weight is not same for every store, move will cause the change of score for two involved store is not equal. So it should be totalSize / totalWeight.
    • sourceScore > targetScore > avgScore, if move, targetScore is still higher than avgScore, and then target store will be scheduled to move some region or leader to other stores with smaller score. But is is better to move some region or leader from source store to store with score lower than avgScore, so it is unnecessary
    • avgScore > sourceScore > targetScore, if move, sourceScore is still lower than avgScore, source store still need other stores to move region or leader to it for closing avgScore, so it is unnecessary.
  2. avgScore is the score when stores in perfect balance state. Based on the weight, we can know the difference of size for current state to be in perfect balance state, and the minimum of them is the largest size of region to be moved. And the tolerantRatio provides a buffer to make the cluster stable, so that we don't need to schedule very frequently.

@Connor1996 Connor1996 changed the title from [WIP] core, schedule: use size as a factor of score to core, schedule: use size as a factor of score Nov 2, 2017

@disksing

This comment has been minimized.

Show comment
Hide comment
@disksing

disksing Nov 6, 2017

Member

LGTM.

Member

disksing commented Nov 6, 2017

LGTM.

@nolouch

nolouch approved these changes Nov 6, 2017

LGTM

@disksing

This comment has been minimized.

Show comment
Hide comment
@disksing

disksing Nov 6, 2017

Member

/run-all-tests

Member

disksing commented Nov 6, 2017

/run-all-tests

Connor1996 added some commits Nov 7, 2017

@@ -21,6 +21,7 @@ import (
type Options interface {
GetLeaderScheduleLimit() uint64
GetRegionScheduleLimit() uint64
GetTolerantSizeRatio() float64

This comment has been minimized.

@siddontang

siddontang Nov 7, 2017

Member

here you use tolerant size, but your TOML file use tolerate size, please choose one.

@siddontang

siddontang Nov 7, 2017

Member

here you use tolerant size, but your TOML file use tolerate size, please choose one.

@siddontang

LGTM

@disksing disksing merged commit 96db717 into pingcap:master Nov 8, 2017

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla Contributor License Agreement is signed.
Details

@Connor1996 Connor1996 deleted the Connor1996:connor/region-size-balance branch Nov 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment