*: new store region score function for balance #1014

Connor1996 · 2018-04-09T04:05:14Z

When storage is in low space, pd just do nothing to balance. This pr introduces a new region score function for balance considering low space as well.

disksing · 2018-04-09T04:53:40Z

server/core/store.go

+		return maxScore - available
+	}
+
+	k := (maxScore - (lowSpaceRatio+highSpaceRatio)*capacity) / ((1 - lowSpaceRatio - highSpaceRatio) * capacity)


add some description about the formula

Does it work well when the capacity is 1T?

disksing · 2018-04-09T04:57:29Z

server/config.go

@@ -352,6 +352,10 @@ type ScheduleConfig struct {
 	MergeScheduleLimit uint64 `toml:"merge-schedule-limit,omitempty" json:"merge-schedule-limit"`
 	// TolerantSizeRatio is the ratio of buffer size for balance scheduler.
 	TolerantSizeRatio float64 `toml:"tolerant-size-ratio,omitempty" json:"tolerant-size-ratio"`
+	// LowSpaceRatio is the lowest available ratio of store which regraded as low space.
+	LowSpaceRatio float64 `toml:"low-space-ratio,omitempty" json:"low-space-ratio"`
+	// HighSpaceRatio is the highest available ratio of store which regraded as high space.


Add some description about how does low space or high space affect balance algorithms.

disksing · 2018-04-09T04:58:04Z

server/core/store.go

@@ -103,15 +104,38 @@ func (s *StoreInfo) DownTime() time.Duration {
 }

 const minWeight = 1e-6
+const maxScore = 10000


I think we can make it 100000.

disksing · 2018-04-09T04:59:57Z

server/core/store.go

-func (s *StoreInfo) LeaderScore() float64 {
-	return float64(s.LeaderSize) / math.Max(s.LeaderWeight, minWeight)
+func (s *StoreInfo) LeaderScore(highSpaceRatio, lowSpaceRatio float64) float64 {
+	return s.score(float64(s.LeaderSize), highSpaceRatio, lowSpaceRatio) / math.Max(s.LeaderWeight, minWeight)


Why does leader score need to consider storage space?

nolouch · 2018-04-19T03:49:24Z

server/core/store.go

+	var score float64
+
+	// lowSpaceRatio shouldn't greater than highSpaceRatio
+	if lowSpaceRatio > highSpaceRatio {


we can check this in config adjustment.

using pd-ctl to set config will not call adjust()

lowSpaceRatio = highSpaceRatio will panic in the formula

when lowSpaceRatio == highSpaceRatio, it is impossible to run into else branch.

disksing · 2018-04-19T05:19:00Z

server/coordinator_test.go

-	store.Stats.Capacity = uint64(1024)
-	store.Stats.Available = store.Stats.Capacity
+	store.Stats.Capacity = 1000 * (1 << 20)
+	store.Stats.Available = store.Stats.Capacity - uint64(store.LeaderSize)


LeaderSize -> RegionSize?

disksing · 2018-04-19T05:21:05Z

server/core/store.go

+		lowSpaceRatio = highSpaceRatio
+	}
+
+	amplification := float64(s.RegionSize) / (float64(s.Stats.GetUsedSize()) / (1 << 20))


Add comment about what does amplification mean?

disksing · 2018-04-19T05:28:57Z

server/schedule/mockcluster.go

 	log "github.com/sirupsen/logrus"
 )

-type mockCluster struct {
-	*schedule.BasicCluster
+// MockCluster is to mock clusterInfo for test use


is used to.

Connor1996 · 2018-04-19T06:20:50Z

I will add check config validation in other PR

disksing

LGTM.

nolouch

LGTM

* server: skip check a region if there is already a pending operator. (#1029) * pdctl, api, schedule: pdctl supports scatter region. (#1028) * add change log (#1024) * scheduler: hot write scheduler randomly select the balance strategy (#1034) * *: new store region score function for balance (#1014) * core: adjust capacity to fit for more cases (#1035) * check config validation (#1036) * *: add metrics for hotspot cache (#1027) * Refine install instructions. (#1041) These extra steps can help new users who are perhaps not quite so familiar with Go setup and conventions. * *: add change log for 2.0 GA (#1038) * *: add change log for ga. * scheduler: adjust metrics (#1042) * fix parse error of config.toml (#1043) * server: resign pd leader when it is not same as etcd leader. (#1039) * server: fix the valid scheduler was deleted when start coordinator (#1045) * server: fix the valid scheduler was deleted when start coordinator

…1.11.1 in /tools/pd-tso-bench (#5990) ref #897, ref #962, ref #969, ref #974, ref #975, ref #976, ref #986, ref prometheus/client_golang#987, ref #987, ref #989, ref #998, ref #1013, ref #1014, ref #1025, ref #1028, ref #1031, ref #1043, ref #1055, ref #1075, ref #1091, ref #1094, ref #1102, ref #1103, ref #1118, ref #1146, ref #1148, ref #1150 Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…1.11.1 in /tests/client (#5992) ref #897, ref #962, ref #969, ref #974, ref #975, ref #976, ref #986, ref #987, ref prometheus/client_golang#987, ref #989, ref #998, ref #1013, ref #1014, ref #1025, ref #1028, ref #1031, ref #1043, ref #1055, ref #1075, ref #1091, ref #1094, ref #1102, ref #1103, ref #1118, ref #1146, ref #1148, ref #1150, ref #4399 Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…1.11.1 in /tests/mcs (#5993) ref #897, ref #962, ref #969, ref #974, ref #975, ref #976, ref #986, ref #987, ref prometheus/client_golang#987, ref #989, ref #998, ref #1013, ref #1014, ref #1025, ref #1028, ref #1031, ref #1043, ref #1055, ref #1075, ref #1091, ref #1094, ref #1102, ref #1103, ref #1118, ref #1146, ref #1148, ref #1150, ref #4399 Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

…1.11.1 in /client (#5991) ref #897, ref #962, ref #969, ref #974, ref #975, ref #976, ref #986, ref #987, ref prometheus/client_golang#987, ref #989, ref #998, ref #1013, ref #1014, ref #1025, ref #1028, ref #1031, ref #1043, ref #1055, ref #1075, ref #1091, ref #1094, ref #1102, ref #1103, ref #1118, ref #1146, ref #1148, ref #1150, ref #4399 Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

Connor1996 added 2 commits April 4, 2018 16:50

use new formula

32d0219

fix test

5b05ce4

Connor1996 changed the title *: new score formula [WIP] *: new score formula Apr 9, 2018

Connor1996 requested review from disksing and nolouch and removed request for disksing April 9, 2018 04:05

disksing reviewed Apr 9, 2018

View reviewed changes

Connor1996 added 13 commits April 10, 2018 16:18

address comment

5fe7c12

adjust should balance

6775f99

Merge branch 'master' into new-formula

a009461

Merge branch 'master' into new-formula

09bafae

add metrics

29499ac

fix leader scheduler

0790027

tiny adjustment

7fe554e

fix balance region

bc3c2e3

fix metrics

9805efd

Merge branch 'master' into new-formula

e92367c

fix bug when region size is zero

3101706

add influence resource size

e60230b

update test

80bc806

Connor1996 changed the title ~~[WIP] *: new score formula~~ *: new score function Apr 18, 2018

add comment

e576ea3

Connor1996 force-pushed the new-formula branch from 7718a55 to e576ea3 Compare April 18, 2018 15:05

Connor1996 added 2 commits April 18, 2018 23:05

Merge branch 'master' into new-formula

6ee7a0d

tiny change

17fdf95

Connor1996 mentioned this pull request Apr 19, 2018

grafana: update pd metrics pingcap/tidb-ansible#356

Merged

nolouch reviewed Apr 19, 2018

View reviewed changes

disksing reviewed Apr 19, 2018

View reviewed changes

address comment

562cd4a

disksing added the priority/P1 The issue has P1 priority. label Apr 19, 2018

reverse space ratio

d6ec4e2

Connor1996 force-pushed the new-formula branch from 47c5f69 to d6ec4e2 Compare April 19, 2018 06:14

Connor1996 added 2 commits April 19, 2018 14:15

Merge branch 'master' into new-formula

1b008df

fix comment

673ec1b

fix test

4f557e0

disksing approved these changes Apr 19, 2018

View reviewed changes

Connor1996 changed the title *: new score function *: new store region score function for balance Apr 19, 2018

nolouch reviewed Apr 19, 2018

View reviewed changes

Connor1996 merged commit e94ccf5 into tikv:master Apr 19, 2018

Connor1996 deleted the new-formula branch April 19, 2018 07:20

disksing pushed a commit to oh-my-tidb/pd that referenced this pull request May 7, 2018

*: new store region score function for balance (tikv#1014)

47a5022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: new store region score function for balance #1014

*: new store region score function for balance #1014

Connor1996 commented Apr 9, 2018 •

edited

disksing Apr 9, 2018

nolouch Apr 9, 2018

disksing Apr 9, 2018

disksing Apr 9, 2018

disksing Apr 9, 2018

nolouch Apr 19, 2018

Connor1996 Apr 19, 2018

nolouch Apr 19, 2018

Connor1996 Apr 19, 2018

disksing Apr 19, 2018

disksing Apr 19, 2018

disksing Apr 19, 2018

Connor1996 commented Apr 19, 2018

disksing left a comment

nolouch left a comment

*: new store region score function for balance #1014

*: new store region score function for balance #1014

Conversation

Connor1996 commented Apr 9, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Connor1996 commented Apr 19, 2018

disksing left a comment

Choose a reason for hiding this comment

nolouch left a comment

Choose a reason for hiding this comment

Connor1996 commented Apr 9, 2018 •

edited