Initially implement distributed consistency with jraft #1020

Linary · 2020-06-07T13:27:48Z

Change-Id: I75671444d48a2851b4d1ad0de8975853dd1f900f

hugegraph-core/src/main/java/com/baidu/hugegraph/StandardHugeGraph.java

...raph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/BackendStoreStateMachine.java

...raph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftNode.java

javeme · 2020-06-08T05:43:57Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftSharedComponent.java

+        this.readIndexExecutor = this.initReadIndexExecutor();
+    }
+
+    public HugeConfig config() {


set protected

use public keep the same with others

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftSharedComponent.java

...raph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/BackendStoreStateMachine.java

codecov · 2020-06-08T06:05:21Z

Codecov Report

Merging #1020 into master will increase coverage by 3.65%.
The diff coverage is 9.73%.

@@             Coverage Diff              @@
##             master    #1020      +/-   ##
============================================
+ Coverage     60.35%   64.01%   +3.65%     
- Complexity     4709     5521     +812     
============================================
  Files           331      347      +16     
  Lines         26997    28520    +1523     
  Branches       3865     4028     +163     
============================================
+ Hits          16295    18257    +1962     
+ Misses         8915     8461     -454     
- Partials       1787     1802      +15

Impacted Files	Coverage Δ	Complexity Δ
...om/baidu/hugegraph/backend/store/BackendStore.java	`80.76% <0.00%> (-6.74%)`	`4.00 <0.00> (ø)`
...hugegraph/backend/store/raft/RaftBackendStore.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...h/backend/store/raft/RaftBackendStoreProvider.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...du/hugegraph/backend/store/raft/RaftException.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...m/baidu/hugegraph/backend/store/raft/RaftNode.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...idu/hugegraph/backend/store/raft/RaftRequests.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...baidu/hugegraph/backend/store/raft/RaftResult.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...ugegraph/backend/store/raft/RaftSharedContext.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...idu/hugegraph/backend/store/raft/StoreClosure.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...ckend/store/raft/StoreCommandRequestProcessor.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
... and 188 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 125b2ef...50e32ff. Read the comment docs.

javeme · 2020-06-09T06:37:30Z

...raph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java

@@ -592,6 +603,16 @@ public boolean hasChanges() {
            return this.batch.count() > 0;
        }

+        @Override
+        public String dataPath() {
+            return dataPath;


this.dataPath

Here is in StdSession class, it's a inner class

javeme · 2020-06-09T06:37:36Z

...raph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java

+
+        @Override
+        public String walPath() {
+            return walPath;


Here is in StdSession class, it's a inner class

javeme · 2020-06-11T07:54:04Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/StoreStateMachine.java

+            } else {
+                // Follower need deserializeMutation data
+                ByteBuffer buffer = iter.getData();
+                command = StoreCommand.fromBytes(buffer.array());


address history comments

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStore.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/StoreStateMachine.java

Change-Id: I75671444d48a2851b4d1ad0de8975853dd1f900f

Change-Id: I037a3011e63008082f48c1c907e4a36ab48328ce

Change-Id: Iabcb4b4f91a72e231145a5c1605ceb4b55820491

Change-Id: Ie1a55f2fed10ead56e067f7b158275e62a3f0d37

Change-Id: I77287d80d78dcdb37dfbddbb9fee980cf5789d91

Change-Id: I4c8e936389a39da8cab04d89b8818960937473d7

Change-Id: I43f61fad49b3c42b3fe145d45d6135386af5cb59

Change-Id: I19f99e43b06b0382be04b7b05a61da5c123477c4

Check connection failed for address: xxx, maybe write overflow! Change-Id: I5bce82d1020ee2adb5b9baf28abbc4d68989ef56

Change-Id: Ibaf3f3726e299dd9e51f6b763308abc799870a15

Change-Id: I5bdec682ec35daead67cbcf92477b0d761d31610

Change-Id: I49b5a6c48053ae0f5a5ebb0dfeb1651da9f155ea

Change-Id: Ibc5e08488b7eda0049eb57fe68a4e124873e35a3

Linary force-pushed the jraft branch from 3856c93 to 20d29a3 Compare June 7, 2020 13:32

javeme requested changes Jun 8, 2020

View reviewed changes

Linary force-pushed the jraft branch from 20d29a3 to 5730fcd Compare June 8, 2020 05:48

Linary force-pushed the jraft branch 3 times, most recently from 8699a72 to 59e815e Compare June 11, 2020 01:51

javeme reviewed Jun 11, 2020

View reviewed changes

Linary force-pushed the jraft branch 5 times, most recently from c1c7c58 to 8f8a33c Compare June 17, 2020 07:00

Linary force-pushed the jraft branch 3 times, most recently from 6d6f2c2 to 131e422 Compare June 22, 2020 07:45

javeme reviewed Jun 22, 2020

View reviewed changes

Linary force-pushed the jraft branch 4 times, most recently from 98b173b to ce3684c Compare June 30, 2020 03:18

Linary force-pushed the jraft branch 4 times, most recently from f0909d8 to 0baf272 Compare July 7, 2020 10:16

Linary force-pushed the jraft branch 4 times, most recently from 4724a72 to 8f213e9 Compare July 22, 2020 07:50

Linary added 9 commits July 22, 2020 15:52

Initially implement distributed consistency with jraft

6dd0856

Change-Id: I75671444d48a2851b4d1ad0de8975853dd1f900f

support save and load snapshot

0f4d7b6

Change-Id: I037a3011e63008082f48c1c907e4a36ab48328ce

Let StoreCommand register and apply at StateMachine

1b247f3

Change-Id: Iabcb4b4f91a72e231145a5c1605ceb4b55820491

merge tx operation into one store command

1da1980

Change-Id: Ie1a55f2fed10ead56e067f7b158275e62a3f0d37

small code optimize

7fcec4d

Change-Id: I77287d80d78dcdb37dfbddbb9fee980cf5789d91

compress task data

27dc4f5

Change-Id: I4c8e936389a39da8cab04d89b8818960937473d7

adjust some raft options, like: disruptor capacity, rpc threadpool size

6c7266b

Change-Id: I43f61fad49b3c42b3fe145d45d6135386af5cb59

let follower can forward request to leader

cae94d5

Change-Id: I19f99e43b06b0382be04b7b05a61da5c123477c4

handle remote exception, let it report backend busy

36ea27b

Check connection failed for address: xxx, maybe write overflow! Change-Id: I5bce82d1020ee2adb5b9baf28abbc4d68989ef56

Linary force-pushed the jraft branch 2 times, most recently from 61ebaf3 to a69b047 Compare July 22, 2020 07:53

add rate limit at StateMachine

4e2c7a8

Change-Id: Ibaf3f3726e299dd9e51f6b763308abc799870a15

Linary force-pushed the jraft branch from a69b047 to 4e2c7a8 Compare July 22, 2020 07:57

Linary added 3 commits July 22, 2020 15:59

Let grizzly-http thread wait raft task applied

eed3d35

Change-Id: I5bdec682ec35daead67cbcf92477b0d761d31610

exclude log4j dependency from jraft

8b45e67

Change-Id: I49b5a6c48053ae0f5a5ebb0dfeb1651da9f155ea

some improve

50e32ff

Change-Id: Ibc5e08488b7eda0049eb57fe68a4e124873e35a3

Linary force-pushed the jraft branch from 22343ab to 50e32ff Compare July 27, 2020 13:25

javeme approved these changes Jul 27, 2020

View reviewed changes

houzhizhen approved these changes Jul 28, 2020

View reviewed changes

houzhizhen merged commit a25000e into master Jul 28, 2020

Linary deleted the jraft branch July 28, 2020 02:53

ingla9988 mentioned this pull request Jun 25, 2021

[Question] 高可用使用咨询 #1517

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initially implement distributed consistency with jraft #1020

Initially implement distributed consistency with jraft #1020

Linary commented Jun 7, 2020

javeme Jun 8, 2020

Linary Jun 12, 2020

codecov bot commented Jun 8, 2020 •

edited

javeme Jun 9, 2020

Linary Jun 12, 2020

javeme Jun 9, 2020

Linary Jun 12, 2020

javeme Jun 11, 2020

Linary Jun 12, 2020

Initially implement distributed consistency with jraft #1020

Initially implement distributed consistency with jraft #1020

Conversation

Linary commented Jun 7, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 8, 2020 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 8, 2020 •

edited