Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify scheduler codes #62

Merged
merged 24 commits into from
Feb 20, 2020
Merged

Simplify scheduler codes #62

merged 24 commits into from
Feb 20, 2020

Conversation

YangKeao
Copy link
Collaborator

  • remove persistent config
  • remove label and related score
  • remove leveldb_kv
  • remove extra filters:
    • snapshotcount
    • pending peer count
    • storage
  • simplify balance region score calculation

And fix related tests.

@YangKeao
remove persistent config
2001214 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove label property
96572c4 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove label & location related score
04cbc53 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove unused codes
5540140 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove leveldb_kv
395e88c 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove snapshotcount filter and peer count filter
a750e49 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove unused code
4c39eea 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove unused cache
244091c 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
change Schedule function signature
8017534 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
simplify balance region source calc
92a11c8 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove unused code
777ac8b 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao YangKeao mentioned this pull request Feb 17, 2020
Closed
8 tasks
Connor1996
Connor1996 reviewed Feb 17, 2020
View reviewed changes
scheduler/server/schedule/range_cluster.go

// GenRangeCluster gets a range cluster by specifying start key and end key.
// The cluster can only know the regions within [startKey, endKey].
func GenRangeCluster(cluster opt.Cluster, startKey, endKey []byte) *RangeCluster {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove RangeCluster totally?

scheduler/server/schedulers/balance_leader.go Outdated
// balanceLeaderRetryLimit is the limit to retry schedule for selected source store and target store.
const balanceLeaderRetryLimit = 10

type balanceLeaderScheduler struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please keep balanceLeaderScheduler though it's not included in the course outline.

scheduler/server/server.go Outdated
@@ -556,76 +536,17 @@ func (s *Server) SetReplicationConfig(cfg config.ReplicationConfig) error {
func (s *Server) SetPDServerConfig(cfg config.PDServerConfig) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove these API too, we don't need to change configs dynamically.

scheduler/server/server.go Outdated
func (s *Server) GetLabelProperty() config.LabelPropertyConfig {
return s.scheduleOpt.LoadLabelPropertyConfig().Clone()
}

// SetClusterVersion sets the version of cluster.
func (s *Server) SetClusterVersion(v string) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please clean all things related with cluster version

@YangKeao
remove dynamic config setting
af1586a 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove push driver
622f975 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove range_cluster
301d1b2 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove cluster_version
b10332b 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove unused function and variable
04b1642 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
format
bf02a8f 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
remove unused functions
40afd90 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Connor1996
Connor1996 reviewed Feb 20, 2020
View reviewed changes
Copy link
Collaborator

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

scheduler/server/cluster_test.go Outdated
@@ -1143,34 +1068,6 @@ func (s *testClusterInfoSuite) TestRegionHeartbeat(c *C) {
}
}

func heartbeatRegions(c *C, cluster *RaftCluster, regions []*core.RegionInfo) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove this?

scheduler/server/cluster_worker_test.go Outdated
@@ -90,42 +88,3 @@ func (s *testClusterWorkerSuite) TestValidRequestRegion(c *C) {
c.Assert(cluster.validRequestRegion(r5), IsNil)
cluster.stop()
}

func (s *testClusterWorkerSuite) TestAskSplit(c *C) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please keep it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! forget to restore it.

@YangKeao
restore some tests
5df3534 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao
Merge remote-tracking branch 'upstream/master' into simplify-schedule…
7f6ccd2 
…r-codes
@YangKeao
format
17dff31 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Connor1996
Connor1996 reviewed Feb 20, 2020
View reviewed changes
scheduler/server/config/config.go Outdated
@@ -419,6 +419,8 @@ func (c *Config) configFromFile(path string) (*toml.MetaData, error) {

// ScheduleConfig is the schedule configuration.
type ScheduleConfig struct {
// SplitMergeInterval is the minimum interval time to permit merge after split.
SplitMergeInterval typeutil.Duration `toml:"split-merge-interval,omitempty" json:"split-merge-interval"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be deleted

@YangKeao
remove SplitMergeInterval
f725d91 
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Connor1996
Connor1996 approved these changes Feb 20, 2020
View reviewed changes
Copy link
Collaborator

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Connor1996 Connor1996 merged commit 3744254 into talent-plan:master Feb 20, 2020
Connor1996 pushed a commit that referenced this pull request Apr 30, 2020
@YangKeao @Connor1996
Simplify scheduler codes (#62)
d0da34f 
* remove persistent config

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove label property

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove label & location related score

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused codes

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove leveldb_kv

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove snapshotcount filter and peer count filter

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused code

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused cache

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* change Schedule function signature

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* simplify balance region source calc

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused code

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove dynamic config setting

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove push driver

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove range_cluster

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove cluster_version

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused function and variable

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused config

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* format

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove unused functions

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* restore some tests

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* format

Signed-off-by: Yang Keao <keao.yang@yahoo.com>

* remove SplitMergeInterval

Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Connor1996 Connor1996 Connor1996 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@YangKeao @Connor1996