[DS-6849][MasterServer] fetch more commands and handle in parallel#6850
Merged
lenboo merged 5 commits intoapache:devfrom Nov 19, 2021
Merged
[DS-6849][MasterServer] fetch more commands and handle in parallel#6850lenboo merged 5 commits intoapache:devfrom
lenboo merged 5 commits intoapache:devfrom
Conversation
37cbc62 to
ceeebb7
Compare
Codecov Report
@@ Coverage Diff @@
## dev #6850 +/- ##
============================================
- Coverage 33.09% 33.06% -0.03%
- Complexity 1621 1622 +1
============================================
Files 433 433
Lines 14290 14310 +20
Branches 1426 1427 +1
============================================
+ Hits 4729 4732 +3
- Misses 9111 9127 +16
- Partials 450 451 +1
Continue to review full report at Codecov.
|
kezhenxu94
reviewed
Nov 15, 2021
...uler-server/src/main/java/org/apache/dolphinscheduler/server/master/config/MasterConfig.java
Outdated
Show resolved
Hide resolved
kezhenxu94
reviewed
Nov 15, 2021
Comment on lines
+24
to
+26
| master-fetch-command-num: 10 | ||
| # master prepare execute thread number to limit handle commands in parallel | ||
| master-pre-exec-threads: 10 |
Member
There was a problem hiding this comment.
Same here, strip the prefix master-
kezhenxu94
reviewed
Nov 15, 2021
...r/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterSchedulerService.java
Show resolved
Hide resolved
added 2 commits
November 16, 2021 16:05
|
SonarCloud Quality Gate failed. |
Member
|
LGTM |
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.









this pr close #6849
Purpose of the pull request
To improve the command handle rate by two parts:
When master slot change,
slotCheckanddeleteCommandWithTrycan ensure that one command will not be processed multiple times.