[Summer of Code] Support switch role for ha service #4236
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## 5.0.0-beta-dledger-controller #4236 +/- ##
===================================================================
+ Coverage 43.27% 43.82% +0.55%
- Complexity 6137 6277 +140
===================================================================
Files 818 826 +8
Lines 57559 58520 +961
Branches 7852 7993 +141
===================================================================
+ Hits 24910 25648 +738
- Misses 29412 29586 +174
- Partials 3237 3286 +49
Continue to review full report at Codecov.
|
RongtongJin
added
module/ha
RongtongJin
reviewed
May 3, 2022
store/src/main/java/org/apache/rocketmq/store/config/MessageStoreConfig.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAClient.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAClient.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/EpochFileCache.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAConnection.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAConnection.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/EpochFileCache.java
Outdated
Show resolved
Hide resolved
store/src/test/java/org/apache/rocketmq/store/ha/EpochFileCacheTest.java
Show resolved
Hide resolved
RongtongJin
reviewed
May 3, 2022
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/EpochFileCache.java
Outdated
Show resolved
Hide resolved
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAConnection.java
Outdated
Show resolved
Hide resolved
RongtongJin
reviewed
May 6, 2022
| localEpochCache.initCacheFromEntries(this.epochCache.getAllEntries()); | ||
| localEpochCache.setLastEpochEntryEndOffset(this.messageStore.getMaxPhyOffset()); | ||
|
|
||
| final long truncateOffset = localEpochCache.findConsistentPoint(masterEpochCache); |
There was a problem hiding this comment.
1.如果因为某些原因（比如日志被删除），找不到主备间的一致的点，应该等待人工处理，而不是继续往下走。
2.如果slave是空，是不是可以直接不走截断流程，这里正确的原因主要是找不到主备间的一致的点currentReportedOffset=-1，然后再reportSlaveMaxOffset被修正成0.
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAConnection.java
Outdated
Show resolved
Hide resolved
RongtongJin
reviewed
May 6, 2022
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAClient.java
Show resolved
Hide resolved
Comment on lines
+190
to
+196
| public long getConfirmOffset() { | ||
| long confirmOffset = this.defaultMessageStore.getMaxPhyOffset(); | ||
| for (HAConnection connection : this.connectionList) { | ||
| confirmOffset = Math.min(confirmOffset, connection.getSlaveAckOffset()); | ||
| } | ||
| return confirmOffset; | ||
| } |
There was a problem hiding this comment.
这个后面还得改，得筛选出在SyncStateSet中的connnection然后比较位点
RongtongJin
approved these changes
May 6, 2022
This was referenced May 10, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
tracking issue: #4330
We want unified log replication, using rocketmq's original HaService instead of dledger mode.
Previously, i have done the following job with @RongtongJin
In this pr, I added a haService - AutoSwitchHAService that can switch role, this ha can cooperate with the controller to achieve master-slave switching
In the follow-up work, I will modify the code of the broker to fully realize the master-slave switch
Brief changelog
The ha service protocol is:
Verifying this change
Follow this checklist to help us incorporate your contribution quickly and easily. Notice,
it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.[ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyleto make sure basic checks pass. Runmvn clean install -DskipITsto make sure unit-test pass. Runmvn clean test-compile failsafe:integration-testto make sure integration-test pass.