improve raft module and test by javeme · Pull Request #1721 · apache/hugegraph

javeme · 2022-01-04T04:27:01Z

implement #1384

codecov · 2022-01-04T04:35:08Z

Codecov Report

Merging #1721 (840e7b9) into master (132582a) will increase coverage by 4.27%.
The diff coverage is 73.91%.

@@             Coverage Diff              @@
##             master    #1721      +/-   ##
============================================
+ Coverage     66.55%   70.83%   +4.27%     
+ Complexity     7085      970    -6115     
============================================
  Files           423      443      +20     
  Lines         35598    37664    +2066     
  Branches       4959     5358     +399     
============================================
+ Hits          23692    26679    +2987     
+ Misses         9505     8297    -1208     
- Partials       2401     2688     +287

Impacted Files	Coverage Δ
...main/java/com/baidu/hugegraph/api/job/TaskAPI.java	`51.02% <0.00%> (ø)`
.../baidu/hugegraph/backend/tx/SchemaTransaction.java	`90.16% <0.00%> (-1.13%)`	⬇️
...ain/java/com/baidu/hugegraph/task/TaskManager.java	`74.82% <ø> (+5.75%)`	⬆️
.../backend/store/raft/rpc/StoreCommandProcessor.java	`65.21% <20.00%> (+65.21%)`	⬆️
...h/backend/store/raft/RaftBackendStoreProvider.java	`67.64% <50.00%> (+67.64%)`	⬆️
...ugegraph/backend/store/raft/RaftSharedContext.java	`79.56% <64.51%> (+79.56%)`	⬆️
...m/baidu/hugegraph/backend/store/raft/RaftNode.java	`46.97% <66.66%> (+46.97%)`	⬆️
...ugegraph/backend/store/raft/StoreStateMachine.java	`62.50% <77.41%> (+62.50%)`	⬆️
...hugegraph/backend/store/raft/RaftBackendStore.java	`80.64% <81.81%> (+80.64%)`	⬆️
...gegraph/backend/cache/CachedSchemaTransaction.java	`91.30% <100.00%> (-0.44%)`	⬇️
... and 97 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 132582a...840e7b9. Read the comment docs.

javeme · 2022-01-04T10:16:41Z

Operations on followers may lead to inconsistent data status?

error when truncate

2022-01-04 17:49:15 [store-backend-executor33] [ERROR] c.b.h.b.s.r.StoreStateMachine - Failed to execute backend command: SNAPSHOT
java.lang.IllegalStateException: Can't apply command for all store at one time
        at com.google.common.base.Preconditions.checkState(Preconditions.java:531) ~[guava-25.1-jre.jar:?]
        at com.baidu.hugegraph.util.E.checkState(E.java:68) ~[hugegraph-common-2.0.1.jar:2.0.1.0]
        at com.baidu.hugegraph.backend.store.raft.StoreStateMachine.applyCommand(StoreStateMachine.java:166) ~[hugegraph-core-0.12.0.jar:0.12.0.0]
        at com.baidu.hugegraph.backend.store.raft.StoreStateMachine.lambda$onApply$1(StoreStateMachine.java:136) ~[hugegraph-core-0.12.0.jar:0.12.0.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_111]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]

Change-Id: I016d3fcc4ab50614afdf452e7c9691ee3cc3c70b

Change-Id: I44e0e3c3cb44b806bd87b2216a339673f26d3166

Change-Id: I319928fc5f26838f7ade8f21a57f955eeca7a6d6

Change-Id: Ib2a87081f7411d4ebfaaef2f8fff0140bb4d3dc0

Change-Id: I65fbed8e18d9b04393aa26f5008397cc9b9fc065

Change-Id: I95913fb9c4a2384492bd8a81c1d94e905dd344e6

Change-Id: I0e9628c26566d019edecc22c8114a0cc9bdeee5b

javeme · 2022-01-10T09:33:15Z

raft test error:

Error:  Tests run: 52, Failures: 3, Errors: 32, Skipped: 0, Time elapsed: 84.22 s <<< FAILURE! - in com.baidu.hugegraph.api.ApiTestSuite
Error:  testAppend(com.baidu.hugegraph.api.IndexLabelApiTest)  Time elapsed: 0.137 s  <<< FAILURE!
java.lang.AssertionError: Response with status 400 and content {"exception":"class com.baidu.hugegraph.exception.ExistedException","message":"The index label 'personByAge' has existed","cause":""} expected:<202> but was:<400>
	at com.baidu.hugegraph.api.IndexLabelApiTest.testAppend(IndexLabelApiTest.java:66)

Error:  testAppend(com.baidu.hugegraph.api.IndexLabelApiTest)  Time elapsed: 0.138 s  <<< ERROR!
com.baidu.hugegraph.HugeException: Failed to list indexlabels

Error:  testCreate(com.baidu.hugegraph.api.IndexLabelApiTest)  Time elapsed: 0.034 s  <<< FAILURE!
java.lang.AssertionError: Response with status 400 and content {"exception":"class com.baidu.hugegraph.exception.ExistedException","message":"The index label 'personByAge' has existed","cause":""} expected:<202> but was:<400>
	at com.baidu.hugegraph.api.IndexLabelApiTest.testCreate(IndexLabelApiTest.java:52)

Error:  testCreate(com.baidu.hugegraph.api.IndexLabelApiTest)  Time elapsed: 0.034 s  <<< ERROR!
com.baidu.hugegraph.HugeException: Failed to list indexlabels

Error:  Tests run: 49, Failures: 0, Errors: 29, Skipped: 0, Time elapsed: 119.686 s <<< FAILURE! - in com.baidu.hugegraph.api.ApiTestSuite
Error:  com.baidu.hugegraph.api.SchemaApiTest  Time elapsed: 9.08 s  <<< ERROR!
com.baidu.hugegraph.HugeException: Failed to list indexlabels

Error:  com.baidu.hugegraph.api.VertexApiTest  Time elapsed: 9.124 s  <<< ERROR!
com.baidu.hugegraph.HugeException: Failed to list indexlabels

Error:  com.baidu.hugegraph.api.EdgeApiTest  Time elapsed: 9.169 s  <<< ERROR!
com.baidu.hugegraph.HugeException: Failed to list indexlabels

Error:  com.baidu.hugegraph.api.TaskApiTest  Time elapsed: 9.214 s  <<< ERROR!
com.baidu.hugegraph.HugeException: Failed to list indexlabels

Change-Id: Ibcdef0f33c9c1127908cc9bfdba5f4fd7f1e309a

Change-Id: Iecce2bd9482a136b85cdca9f01615a701fcf1af9

This reverts commit c5fb178.

Change-Id: I8b9cfd1d10c350273dbde9416f3c3439620e44c3

Change-Id: Ie314bbb51091f6cb8191d06d356397ae56bde885

Change-Id: I3e6ca5f24e38eeb9c8b318e996a0697de30b3a1c

javeme · 2022-01-11T12:04:39Z

how to upload multiple report files to codecov:
codecov/codecov-action#47

Change-Id: I43421d7d5a2285d81968bbdd258364d9018c7b21

imbajin · 2022-01-12T07:27:07Z

hugegraph-test/src/main/java/com/baidu/hugegraph/api/GremlinApiTest.java

+                + "\"gremlin\":\"hugegraph.backendStoreFeatures()"
+                + "                       .supportsSharedStorage();\","


consider keep them in one line maybe (same as older like line 70, friendly to read)

imbajin · 2022-01-12T07:28:22Z

hugegraph-test/src/main/java/com/baidu/hugegraph/api/GremlinApiTest.java

+                + "  if (!hugegraph.backendStoreFeatures()"
+                + "                .supportsSharedStorage())"
+                + "    return;"


merge to one line maybe?

imbajin · 2022-01-12T07:28:54Z

hugegraph-test/src/main/java/com/baidu/hugegraph/api/GremlinApiTest.java

        body = "{"
                + "\"gremlin\":\"hugegraph.serverStarted("
-                + "IdGenerator.of('server1'), NodeRole.MASTER)\","
+                + "              IdGenerator.of('server1'), NodeRole.MASTER)\","


* improve raft moddule * add raft-server test * set safe_read=true and use_snapshot=false * fix leader not wait for apply-task * add auth support to raft-tools.sh * don't clear non-shared-storage backend * set task timer interval from 3s to 1s * improve CachedSchemaTransaction * improve update schema status * remove truncate from project test * fix codecov: api test report not been uploaded * improve task cancel() test with cancelled status Change-Id: I016d3fcc4ab50614afdf452e7c9691ee3cc3c70b

javeme force-pushed the raft-improve branch 8 times, most recently from 6e2c2b5 to de7aebb Compare January 4, 2022 08:54

javeme added 7 commits January 4, 2022 18:22

improve raft moddule

2d48493

Change-Id: I016d3fcc4ab50614afdf452e7c9691ee3cc3c70b

add raft-server test

f882651

Change-Id: I44e0e3c3cb44b806bd87b2216a339673f26d3166

set safe_read=true and use_snapshot=false

39a750f

Change-Id: I319928fc5f26838f7ade8f21a57f955eeca7a6d6

fix leader not wait for apply-task

9204ab1

Change-Id: Ib2a87081f7411d4ebfaaef2f8fff0140bb4d3dc0

add auth support to raft-tools.sh

470872b

Change-Id: I65fbed8e18d9b04393aa26f5008397cc9b9fc065

don't clear non-shared-storage backend

a31a667

Change-Id: I95913fb9c4a2384492bd8a81c1d94e905dd344e6

set task timer interval from 3s to 1s

ebe9960

Change-Id: I0e9628c26566d019edecc22c8114a0cc9bdeee5b

javeme force-pushed the raft-improve branch from de7aebb to ebe9960 Compare January 6, 2022 14:58

javeme force-pushed the raft-improve branch from 59ecfcf to 783a9c6 Compare January 10, 2022 13:05

javeme added 2 commits January 11, 2022 14:51

improve CachedSchemaTransaction

022bcdf

Change-Id: Ibcdef0f33c9c1127908cc9bfdba5f4fd7f1e309a

debug exist index label

c5fb178

Change-Id: Iecce2bd9482a136b85cdca9f01615a701fcf1af9

javeme force-pushed the raft-improve branch from 15a0669 to dfc9f1a Compare January 11, 2022 07:19

javeme added 3 commits January 11, 2022 16:15

Revert "debug exist index label"

4e92a46

This reverts commit c5fb178.

improve update schema status

08c5959

Change-Id: I8b9cfd1d10c350273dbde9416f3c3439620e44c3

remove truncate from project test

3f65c04

Change-Id: Ie314bbb51091f6cb8191d06d356397ae56bde885

javeme force-pushed the raft-improve branch from dfc9f1a to 3f65c04 Compare January 11, 2022 08:47

fix codecov: api test report not been uploaded

2820f95

Change-Id: I3e6ca5f24e38eeb9c8b318e996a0697de30b3a1c

improve task cancel() test with cancelled status

840e7b9

Change-Id: I43421d7d5a2285d81968bbdd258364d9018c7b21

Linary approved these changes Jan 12, 2022

View reviewed changes

imbajin reviewed Jan 12, 2022

View reviewed changes

imbajin approved these changes Jan 12, 2022

View reviewed changes

javeme merged commit ace6053 into master Jan 12, 2022

javeme deleted the raft-improve branch January 12, 2022 08:01

javeme mentioned this pull request Feb 16, 2022

Integrate rpc and raft module CI #1384

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

improve raft module and test#1721

improve raft module and test#1721
javeme merged 14 commits intomasterfrom
raft-improve

javeme commented Jan 4, 2022 •

edited

Loading

Uh oh!

codecov bot commented Jan 4, 2022 •

edited

Loading

Uh oh!

javeme commented Jan 4, 2022 •

edited

Loading

Uh oh!

javeme commented Jan 10, 2022 •

edited

Loading

Uh oh!

javeme commented Jan 11, 2022

Uh oh!

imbajin Jan 12, 2022

Uh oh!

imbajin Jan 12, 2022

Uh oh!

imbajin Jan 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		+ "\"gremlin\":\"hugegraph.backendStoreFeatures()"
		+ " .supportsSharedStorage();\","

Comments

Conversation

javeme commented Jan 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

javeme commented Jan 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

javeme commented Jan 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

javeme commented Jan 11, 2022

Uh oh!

imbajin Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

imbajin Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

imbajin Jan 12, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

javeme commented Jan 4, 2022 •

edited

Loading

codecov bot commented Jan 4, 2022 •

edited

Loading

javeme commented Jan 4, 2022 •

edited

Loading

javeme commented Jan 10, 2022 •

edited

Loading