[ISSUE #4813] Add elect policy for controller #4809

TheR1sing3un · 2022-08-10T17:35:11Z

add epoch and maxOffset in heartbeat.
refactor elect logic, now we
elect a new master by elect policy(can expand).
add some unit tests

Make sure set the target branch to develop

What is the purpose of the change

issue link: #4813

Brief changelog

XX

Verifying this change

XXXX

Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

1. add epoch and maxOffset in heartbeat. 2. refactor elect logic, now we elect a new master by elect policy(can expand). 3. add some unit tests

TheR1sing3un · 2022-08-10T17:35:57Z

@RongtongJin Hi~ Please check! Thks!🤣

RongtongJin · 2022-08-11T00:55:18Z

Hi @TheR1sing3un It would be better to open another issue to record this pull request.

...in/java/org/apache/rocketmq/common/protocol/header/namesrv/BrokerHeartbeatRequestHeader.java

controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java

controller/src/main/java/org/apache/rocketmq/controller/pojo/BrokerLiveInfo.java

controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java

lizhanhui · 2022-08-11T06:41:50Z

Is there an issue related? Fix the title following the template guideline

TheR1sing3un · 2022-08-11T06:44:23Z

Is there an issue related? Fix the title following the template guideline

OK, I will open an issue to record this pr soon.

broker/src/main/java/org/apache/rocketmq/broker/BrokerController.java

...in/java/org/apache/rocketmq/common/protocol/header/namesrv/BrokerHeartbeatRequestHeader.java

controller/src/main/java/org/apache/rocketmq/controller/elect/impl/DefaultElectPolicy.java

controller/src/main/java/org/apache/rocketmq/controller/elect/ElectPolicy.java

hzh0425 · 2022-08-11T14:28:08Z

wow, cool

1. refactor some code and format some code

hzh0425 · 2022-08-13T07:32:37Z

Pls fix the misspell and ci

1. fix typo in ReplicasInfoManager

1.fix wrong method call in ReplicasManagerTest#before

1. fix wrong call in ReplicasManagerTest#before

RongtongJin

[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java:20:8: Unused import - java.util.Collections. [UnusedImports]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java:27:8: Unused import - java.util.function.Predicate. [UnusedImports]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java:46:8: Unused import - org.apache.rocketmq.controller.elect.impl.DefaultElectPolicy. [UnusedImports]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:38:8: Unused import - java.util.function.BiFunction. [UnusedImports]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:60:8: Unused import - org.apache.rocketmq.controller.BrokerLiveInfo. [UnusedImports]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:161: 'lambda arguments' has incorrect indentation level 16, expected level should be 12. [Indentation]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:167: 'lambda arguments' has incorrect indentation level 16, expected level should be 12. [Indentation]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:173: 'lambda arguments' has incorrect indentation level 16, expected level should be 12. [Indentation]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:179: 'lambda arguments' has incorrect indentation level 16, expected level should be 12. [Indentation]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java:186: 'lambda arguments' has incorrect indentation level 16, expected level should be 12. [Indentation]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DefaultBrokerHeartbeatManager.java:139:13: '}' is not followed by whitespace. [WhitespaceAround]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/impl/DefaultBrokerHeartbeatManager.java:139:14: 'else' is not preceded with whitespace. [WhitespaceAround]
[ERROR] /home/travis/build/apache/rocketmq/controller/src/main/java/org/apache/rocketmq/controller/BrokerLiveInfo.java:22:8: Unused import - java.util.Objects. [UnusedImports]
Audit done.

Hi @TheR1sing3un you can refer to this page to format your code.

controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java

1. fix the bug about ReElectMaster 2. fix some typo

1. fix typo by checkstyle

…before 1. fix invalid usage of "any()" in DLedegerControllerTest#before

RongtongJin · 2022-08-15T00:55:17Z

Failed tests:
DLedgerControllerTest.testEnableElectUnCleanMaster:253 Values should be different. Actual: 127.0.0.1:9000

1. fix some wrong usage in test 2. refactor some field type to avoid NPE

codecov-commenter · 2022-08-15T04:34:22Z

Codecov Report

Merging #4809 (07100c2) into develop (8dd07c8) will increase coverage by 0.02%.
The diff coverage is 54.41%.

@@              Coverage Diff              @@
##             develop    #4809      +/-   ##
=============================================
+ Coverage      44.94%   44.96%   +0.02%     
- Complexity      7640     7661      +21     
=============================================
  Files            980      982       +2     
  Lines          68135    68222      +87     
  Branches        9014     9022       +8     
=============================================
+ Hits           30622    30678      +56     
- Misses         33729    33749      +20     
- Partials        3784     3795      +11

Impacted Files	Coverage Δ
...a/org/apache/rocketmq/broker/BrokerController.java	`46.43% <0.00%> (+2.54%)`	⬆️
...l/header/namesrv/BrokerHeartbeatRequestHeader.java	`0.00% <0.00%> (ø)`
...oller/RegisterBrokerToControllerRequestHeader.java	`0.00% <0.00%> (ø)`
...ketmq/store/ha/autoswitch/AutoSwitchHAService.java	`56.06% <0.00%> (-0.29%)`	⬇️
...org/apache/rocketmq/broker/out/BrokerOuterAPI.java	`18.60% <16.66%> (-0.12%)`	⬇️
...controller/impl/DefaultBrokerHeartbeatManager.java	`71.26% <44.44%> (-9.23%)`	⬇️
...org/apache/rocketmq/controller/BrokerLiveInfo.java	`53.65% <53.65%> (ø)`
...he/rocketmq/broker/controller/ReplicasManager.java	`44.97% <60.00%> (+0.08%)`	⬆️
.../apache/rocketmq/controller/ControllerManager.java	`75.00% <75.00%> (ø)`
...etmq/controller/elect/impl/DefaultElectPolicy.java	`75.00% <75.00%> (ø)`
... and 30 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

coveralls · 2022-08-15T04:38:20Z

Coverage decreased (-0.2%) to 49.037% when pulling 07100c2 on TheR1sing3un:develop into 8dd07c8 on apache:develop.

coveralls · 2022-08-15T04:38:21Z

Coverage decreased (-0.2%) to 49.037% when pulling 07100c2 on TheR1sing3un:develop into 8dd07c8 on apache:develop.

RongtongJin · 2022-08-15T06:28:37Z

@hzh0425 @mxsm Plz help review

hzh0425

LGTM

feat(controller): add elect policy

f1461a2

1. add epoch and maxOffset in heartbeat. 2. refactor elect logic, now we elect a new master by elect policy(can expand). 3. add some unit tests

RongtongJin requested a review from hzh0425 August 11, 2022 00:55

RongtongJin reviewed Aug 11, 2022

View reviewed changes

mxsm reviewed Aug 11, 2022

View reviewed changes

RongtongJin changed the title ~~feat(controller): add elect policy~~ [ISSUE #4813] Add elect policy for controller Aug 12, 2022

TheR1sing3un and others added 2 commits August 13, 2022 02:10

refactor(controller): refactor some code and format some code

7885ceb

1. refactor some code and format some code

Merge branch 'develop' into develop

9cfbcff

TheR1sing3un added 3 commits August 13, 2022 22:32

fix typo in ReplicasInfoManager

c72e685

1. fix typo in ReplicasInfoManager

fix wrong method call in ReplicasManagerTest#before

d58951d

1.fix wrong method call in ReplicasManagerTest#before

fix wrong call in ReplicasManagerTest#before

838670b

1. fix wrong call in ReplicasManagerTest#before

RongtongJin reviewed Aug 13, 2022

View reviewed changes

controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java Show resolved Hide resolved

TheR1sing3un added 3 commits August 14, 2022 19:51

fix(controller): fix the bug about ReElectMaster

c5bcb2c

1. fix the bug about ReElectMaster 2. fix some typo

style(controller): fix typo by checkstyle

b46ea70

1. fix typo by checkstyle

test(broker): fix invalid usage of "any()" in DLedegerControllerTest#…

ee6e857

…before 1. fix invalid usage of "any()" in DLedegerControllerTest#before

fix(controller): fix some wrong usage in test

07100c2

1. fix some wrong usage in test 2. refactor some field type to avoid NPE

RongtongJin approved these changes Aug 15, 2022

View reviewed changes

hzh0425 approved these changes Aug 15, 2022

View reviewed changes

RongtongJin merged commit c5f8071 into apache:develop Aug 16, 2022

[ISSUE #4813] Add elect policy for controller #4809

[ISSUE #4813] Add elect policy for controller #4809

Uh oh!

Conversation

TheR1sing3un commented Aug 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief changelog

Verifying this change

Uh oh!

TheR1sing3un commented Aug 10, 2022

Uh oh!

RongtongJin commented Aug 11, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lizhanhui commented Aug 11, 2022

Uh oh!

TheR1sing3un commented Aug 11, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hzh0425 commented Aug 11, 2022

Uh oh!

hzh0425 commented Aug 13, 2022

Uh oh!

RongtongJin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RongtongJin commented Aug 15, 2022

Uh oh!

codecov-commenter commented Aug 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coveralls commented Aug 15, 2022

Uh oh!

coveralls commented Aug 15, 2022

Uh oh!

RongtongJin commented Aug 15, 2022

Uh oh!

hzh0425 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

TheR1sing3un commented Aug 10, 2022 •

edited

Loading

codecov-commenter commented Aug 15, 2022 •

edited

Loading