mcs: make scheduling server support checker and scheduler http interface #7131

lhy1024 · 2023-09-21T12:57:06Z

What problem does this PR solve?

Issue Number: Ref #5839

What is changed and how does it work?

It can be reviewed after #7090 and #7169

Check List

Tests

Unit test

Release note

None.

ti-chi-bot · 2023-09-21T12:57:09Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

JmPotato
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

ti-chi-bot · 2023-09-21T12:57:09Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Signed-off-by: lhy1024 <admin@liudos.us>

codecov · 2023-09-27T17:34:27Z

Codecov Report

Merging #7131 (d19578d) into master (849d80d) will increase coverage by 0.01%.
Report is 4 commits behind head on master.
The diff coverage is 58.54%.

@@            Coverage Diff             @@
##           master    #7131      +/-   ##
==========================================
+ Coverage   74.58%   74.60%   +0.01%     
==========================================
  Files         441      441              
  Lines       47292    47365      +73     
==========================================
+ Hits        35275    35336      +61     
+ Misses       8940     8936       -4     
- Partials     3077     3093      +16

Flag	Coverage Δ
unittests	`74.60% <58.54%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

rleungx · 2023-09-28T02:33:38Z

pkg/mcs/scheduling/server/apis/v1/api.go

@@ -120,12 +119,15 @@ func NewService(srv *scheserver.Service) *Service {
 func (s *Service) RegisterSchedulersRouter() {
 	router := s.root.Group("schedulers")
 	router.GET("", getSchedulers)
+	router.GET("/diagnostic/:name", getDiagnosticResult)


We don't need it for now.

why? It can pass tests

This feature is not widely used, to reduce the problem, I think we can hold it instead of expose it.

I think it's best to keep forwarding, one reason is that it will make matching rules simple, and another reason is to avoid us missing an interface or creating incompatibility issues

rleungx · 2023-09-28T02:37:45Z

pkg/mcs/scheduling/server/apis/v1/api.go

@@ -120,12 +119,15 @@ func NewService(srv *scheserver.Service) *Service {
 func (s *Service) RegisterSchedulersRouter() {
 	router := s.root.Group("schedulers")
 	router.GET("", getSchedulers)
+	router.GET("/diagnostic/:name", getDiagnosticResult)
+	router.POST("/:name", pauseOrResumeScheduler)


Shall we split it into two APIs?

No, we only support one-to-one forwarding at this time.

What I mean is since we use the new framework for scheduling service, we should make it more standard. The previous one is not good enough, so we should make it more clear.

If we want to achieve one-to-two forwarding, i.e. post scheduler/{name} is at the api server and forwards to post scheduler/{name}/pause and post scheduler/{name}/resume, we may need to add a check for body to the match rule.

We should make the new API more clear.

If you don't want to change it and since it is only used by forwarding now, I'm ok with the current status. But in the future, it's better to separate these actions.

ok, I will add TODO about it.

rleungx · 2023-09-28T02:41:07Z

tests/integrations/mcs/scheduling/api_test.go

@@ -159,10 +159,17 @@ func (suite *apiTestSuite) TestAPIForward() {
 	pauseArgs, err := json.Marshal(input)
 	suite.NoError(err)
 	err = testutil.CheckPostJSON(testDialClient, fmt.Sprintf("%s/%s", urlPrefix, "checker/merge"), pauseArgs,
-		testutil.StatusOK(re), testutil.WithoutHeader(re, apiutil.PDRedirectorHeader))


Why change it?

Because we should allow to redirect pause/resume checker

lhy1024 · 2023-10-08T07:41:53Z

/merge

ti-chi-bot · 2023-10-08T07:41:55Z

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-10-08T07:41:56Z

This pull request has been accepted and is ready to merge.

Commit hash: 2fad820

ti-chi-bot · 2023-10-08T07:42:09Z

@lhy1024: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 · 2023-10-08T09:00:24Z

/merge

ti-chi-bot · 2023-10-08T09:00:25Z

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-10-08T09:00:27Z

This pull request has been accepted and is ready to merge.

Commit hash: d19578d

…ace (tikv#7131) ref tikv#5839 Signed-off-by: lhy1024 <admin@liudos.us>

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Sep 21, 2023

ti-chi-bot bot requested review from disksing and HunDunDM September 21, 2023 12:57

ti-chi-bot bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 21, 2023

rleungx mentioned this pull request Sep 27, 2023

Separate scheduling into an independent service #5839

Closed

lhy1024 changed the title ~~mcs: add checker, scheduler and hot status http forward~~ mcs: make scheduling server support checker and scheduler http interface Sep 27, 2023

mcs: make scheduling server support checker and scheduler http interface

6719a08

Signed-off-by: lhy1024 <admin@liudos.us>

lhy1024 force-pushed the sche-redirect3 branch from 3f25d52 to 6719a08 Compare September 27, 2023 17:05

lhy1024 marked this pull request as ready for review September 27, 2023 17:17

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 27, 2023

lhy1024 requested review from JmPotato and rleungx and removed request for disksing and HunDunDM September 27, 2023 17:17

rleungx reviewed Sep 28, 2023

View reviewed changes

rleungx approved these changes Oct 8, 2023

View reviewed changes

ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 8, 2023

JmPotato approved these changes Oct 8, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 8, 2023

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 8, 2023

add TODO

00950cb

Signed-off-by: lhy1024 <admin@liudos.us>

Merge branch 'master' of github.com:tikv/pd into sche-redirect3

d19578d

lhy1024 force-pushed the sche-redirect3 branch from 56451c9 to d19578d Compare October 8, 2023 08:14

ti-chi-bot bot removed the status/can-merge Indicates a PR has been approved by a committer. label Oct 8, 2023

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 8, 2023

ti-chi-bot bot merged commit edf1f95 into tikv:master Oct 8, 2023
24 of 26 checks passed

lhy1024 deleted the sche-redirect3 branch October 8, 2023 14:48

lhy1024 mentioned this pull request Oct 9, 2023

mcs: forward request to scheduling server #7178

Closed

25 tasks

lhy1024 mentioned this pull request Nov 27, 2023

api: inconsistent results for empty scheduler #7452

Closed

rleungx pushed a commit to rleungx/pd that referenced this pull request Dec 1, 2023

mcs: make scheduling server support checker and scheduler http interf…

c547876

…ace (tikv#7131) ref tikv#5839 Signed-off-by: lhy1024 <admin@liudos.us>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mcs: make scheduling server support checker and scheduler http interface #7131

mcs: make scheduling server support checker and scheduler http interface #7131

lhy1024 commented Sep 21, 2023 •

edited

Loading

ti-chi-bot bot commented Sep 21, 2023 •

edited

Loading

ti-chi-bot bot commented Sep 21, 2023

codecov bot commented Sep 27, 2023 •

edited

Loading

rleungx Sep 28, 2023

lhy1024 Sep 28, 2023

rleungx Sep 28, 2023

lhy1024 Sep 28, 2023

rleungx Sep 28, 2023

lhy1024 Sep 28, 2023

rleungx Sep 28, 2023

lhy1024 Sep 28, 2023

rleungx Oct 8, 2023

rleungx Oct 8, 2023

lhy1024 Oct 8, 2023

rleungx Sep 28, 2023

lhy1024 Sep 28, 2023

lhy1024 commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

lhy1024 commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

mcs: make scheduling server support checker and scheduler http interface #7131

mcs: make scheduling server support checker and scheduler http interface #7131

Conversation

lhy1024 commented Sep 21, 2023 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Sep 21, 2023 • edited Loading

ti-chi-bot bot commented Sep 21, 2023

codecov bot commented Sep 27, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lhy1024 commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

lhy1024 commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

ti-chi-bot bot commented Oct 8, 2023

lhy1024 commented Sep 21, 2023 •

edited

Loading

ti-chi-bot bot commented Sep 21, 2023 •

edited

Loading

codecov bot commented Sep 27, 2023 •

edited

Loading