-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[improve] [broker] improve the restart broker can load bundle as one of the best brokers when selectBroker in LeastResourceUsageWithWeight #16766
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
gaozhangmin
merged 1 commit into
apache:master
from
Nicklee007:improve-historyusage-clean-for-restart-broker
Dec 9, 2022
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
e386176 to
8cf80d6
Compare
bd24ac3 to
3a4ae32
Compare
b5f8001 to
cb48cba
Compare
cb48cba to
c83b387
Compare
|
The pr had no activity for 30 days, mark with Stale label. |
c83b387 to
3c0c603
Compare
Contributor
Author
|
/pulsarbot run-failure-checks |
HQebupt
approved these changes
Dec 8, 2022
Contributor
HQebupt
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
gaozhangmin
approved these changes
Dec 8, 2022
Codecov Report
@@ Coverage Diff @@
## master #16766 +/- ##
============================================
+ Coverage 44.79% 45.20% +0.41%
- Complexity 10026 10817 +791
============================================
Files 703 763 +60
Lines 68808 73575 +4767
Branches 7375 7912 +537
============================================
+ Hits 30822 33260 +2438
- Misses 34378 36592 +2214
- Partials 3608 3723 +115
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Demogorgon314
pushed a commit
to Demogorgon314/pulsar
that referenced
this pull request
Dec 26, 2022
…of the best brokers when selectBroker in LeastResourceUsageWithWeight (apache#16766) Co-authored-by: nicklixinyang <nicklixinyang@didiglobal.com>
Demogorgon314
pushed a commit
to Demogorgon314/pulsar
that referenced
this pull request
Dec 29, 2022
…of the best brokers when selectBroker in LeastResourceUsageWithWeight (apache#16766) Co-authored-by: nicklixinyang <nicklixinyang@didiglobal.com>
lifepuzzlefun
pushed a commit
to lifepuzzlefun/pulsar
that referenced
this pull request
Jan 10, 2023
…of the best brokers when selectBroker in LeastResourceUsageWithWeight (apache#16766) Co-authored-by: nicklixinyang <nicklixinyang@didiglobal.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
As the
brokerAvgResourceUsageWithWeightstored the broker's history resource usage, if a broker is high load before restart, after the broker restart always show a high load history resource usage, andloadBalancerHistoryResourcePercentage = 0.9, which will cause the first few times the broker can not be select as best broker to load bundles. Only whenselectBrokerForAssignmentinvoke few time's , thebrokerAvgResourceUsageWithWeightcould be neutralized by the current resourceUsage.But bundles unload is not a frequently operator if not disable
doLoadShedding,it will wait a long time; So if a broker restarted we can clean the brokers history resource usage to guaranty the no bundles or no msgRate broker can be select as the one of best brokers.pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LeastResourceUsageWithWeight.java
Lines 99 to 100 in bc94643
Modifications
Documentation
doc-not-needed(Please explain why)
Matching PR in forked repository
PR in forked repository:Nicklee007#3