Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedule: actively push operator #1536

Merged
merged 11 commits into from May 19, 2019

Conversation

@nolouch
Copy link
Member

nolouch commented May 15, 2019

What problem does this PR solve?

For tikv/tikv#4591 and speed up scheduling, we need actively push the operator.

What is changed and how it works?

use a heap to get the needed push operator.

Check List

Tests

  • Unit test
  • Manualy test
    TiKV use this branch: tikv/tikv#4684
    Without the improve:
    Screen Shot 2019-05-16 at 12 51 58 AM
    The branch:
    Screen Shot 2019-05-16 at 12 53 48 AM
nolouch added 2 commits May 15, 2019
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch added the WIP label May 15, 2019
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch force-pushed the nolouch:active-operator branch from a0c116c to b8998b1 May 15, 2019
nolouch added 2 commits May 16, 2019
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
server/coordinator.go Outdated Show resolved Hide resolved
server/coordinator.go Show resolved Hide resolved
server/coordinator.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_queue.go Outdated Show resolved Hide resolved
server/schedule/operator_queue.go Outdated Show resolved Hide resolved
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch removed the WIP label May 16, 2019
@codecov-io

This comment has been minimized.

Copy link

codecov-io commented May 16, 2019

Codecov Report

❗️ No coverage uploaded for pull request base (master@6febb24). Click here to learn what that means.
The diff coverage is 91.46%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #1536   +/-   ##
=========================================
  Coverage          ?   67.75%           
=========================================
  Files             ?      159           
  Lines             ?    15740           
  Branches          ?        0           
=========================================
  Hits              ?    10664           
  Misses            ?     4112           
  Partials          ?      964
Impacted Files Coverage Δ
tests/cluster.go 84.27% <100%> (ø)
server/schedule/operator_queue.go 100% <100%> (ø)
server/cluster_worker.go 48.33% <100%> (ø)
server/coordinator.go 85.39% <100%> (ø)
server/schedule/operator_controller.go 92.17% <86.53%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6febb24...93d3142. Read the comment docs.

@nolouch nolouch requested review from rleungx and disksing May 16, 2019
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_queue.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Outdated Show resolved Hide resolved
server/schedule/operator_controller_test.go Outdated Show resolved Hide resolved
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch force-pushed the nolouch:active-operator branch from 7608f63 to 8016601 May 17, 2019
@nolouch nolouch force-pushed the nolouch:active-operator branch from 93d3142 to 8016601 May 17, 2019
Copy link
Contributor

shafreeck left a comment

I think the main logic in this PR is implementing a timer with a priority queue, so why not use time.Timer instead?

server/coordinator.go Outdated Show resolved Hide resolved
server/schedule/operator_controller.go Show resolved Hide resolved
server/schedule/operator_queue.go Show resolved Hide resolved
nolouch added 2 commits May 17, 2019
Signed-off-by: nolouch <nolouch@gmail.com>
Copy link
Contributor

shafreeck left a comment

LGTM

@nolouch

This comment has been minimized.

Copy link
Member Author

nolouch commented May 17, 2019

/run-all-tests

@nolouch

This comment has been minimized.

Copy link
Member Author

nolouch commented May 19, 2019

/ok-to-test

@nolouch nolouch merged commit b6150ca into pingcap:master May 19, 2019
8 checks passed
8 checks passed
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
idc-jenkins-ci-pd/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-pd/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-pd/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci/build Jenkins job succeeded.
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@nolouch nolouch deleted the nolouch:active-operator branch May 19, 2019
nolouch added a commit that referenced this pull request May 23, 2019
* tiny clean up code (#1526)

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* config:  enable use region storage by default (#1524)

* config: default enable use region storage

* schedulers: let hot region balance not affect by balance-region-scheduler-limit (#1522)

Signed-off-by: nolouch <nolouch@gmail.com>

* log: do not add stack to error log (#1532)

* *: replace gofail with pingcap/failpoint (#1534)

* *: replace gofail with pingcap/failpoint

Signed-off-by: disksing <i@disksing.com>

* fix typo

Signed-off-by: disksing <i@disksing.com>

* fix typo

Signed-off-by: disksing <i@disksing.com>

* Update server/cluster_test.go

Co-Authored-By: Lonng <chris@lonng.org>

* Update server/tso.go

Co-Authored-By: Lonng <chris@lonng.org>

* Update server/tso.go

Co-Authored-By: Lonng <chris@lonng.org>

* check enable failpoint result

Signed-off-by: disksing <i@disksing.com>

* Update server/cluster.go

Co-Authored-By: Lonng <chris@lonng.org>

* server: set timeout for MoveLeader (#1533)

* server: set timeout for MoveLeader

Signed-off-by: disksing <i@disksing.com>

* client, server: add ScanRegions gRPC protocol support (#1535)

* client, server: support ScanRegions gRPC protocol

Signed-off-by: disksing <i@disksing.com>

* schedule: actively push operator  (#1536)

* schedule: actively push operator

Signed-off-by: nolouch <nolouch@gmail.com>

* *: update some dead links (#1543)

* update links

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* Add windows build script (#1540)

Signed-off-by: Ana Hobden <operator@hoverbear.org>

* fix operator timeout metrics (#1541)

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* *:Rollback config in store when kv.persist failed (#1476)

* tests: independent region sync test (#1545)

* tests: independent syncer region test

Signed-off-by: nolouch <nolouch@gmail.com>

* schedule: operator limit for stores (#1474)

* add store limit for scheduling

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* statistic: add the statistic of flow (#1548)

* store_statistic: add the statistic of flow

Signed-off-by: nolouch <nolouch@gmail.com>
nolouch added a commit that referenced this pull request Aug 16, 2019
* schedule: actively push operator

Signed-off-by: nolouch <nolouch@gmail.com>
nolouch added a commit that referenced this pull request Aug 21, 2019
* schedule: actively push operator  (#1536)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.