Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lock graph when close() #1104

Merged
merged 6 commits into from
Jul 24, 2020
Merged

lock graph when close() #1104

merged 6 commits into from
Jul 24, 2020

Conversation

javeme
Copy link
Contributor

@javeme javeme commented Jul 22, 2020

also fix scheduleOrExecuteJob and closeScheduler dead lock:
when main thread catched lock in closeScheduler() then call closeSchedulerTx()
and wait for schedulerExecutor, at the same time scheduler thread call
scheduleOrExecuteJob() but can't get synchronized lock, caused dead lock.

fix #1075
Change-Id: I47e7d4bbb87dd1330dd33616b6d2c33b2a39a89e

when main thread catched lock in closeScheduler() then call closeSchedulerTx()
and wait for schedulerExecutor, at the same time scheduler thread call
scheduleOrExecuteJob() but can't get synchronized lock, caused dead lock.

Change-Id: I4596c87ab047ee9848566de7e9f0ec79fb1053be
fix #1075
Change-Id: I47e7d4bbb87dd1330dd33616b6d2c33b2a39a89e
Change-Id: Ib94f8ba15a3f8cbbcffe5ec48f16df2a5020fe79
Change-Id: I400a2e32d162a0b2b6968b8361f8fb995eb940e6
Change-Id: Iebcf0c9aa814f6007c9f5734eb3a2fdbf4b6989c
Change-Id: If72a393cc5614e063b4cd02bfd725a011db633ad
@javeme
Copy link
Contributor Author

javeme commented Jul 23, 2020

calling initialized() cause reopen tx when graph is closed hugegraph@dc7b924#diff-a7ddfcbed05d0ca6b726444d2184723fR304

[ERROR] Tests run: 673, Failures: 3, Errors: 1, Skipped: 13, Time elapsed: 868.376 s <<< FAILURE! - in com.baidu.hugegraph.core.CoreTestSuite
[ERROR] testCreateGraphsWithDifferentNameDifferentBackends(com.baidu.hugegraph.core.MultiGraphsTest)  Time elapsed: 48.688 s  <<< FAILURE!
java.lang.AssertionError: java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'g1'
	at com.baidu.hugegraph.core.MultiGraphsTest.destoryGraphs(MultiGraphsTest.java:352)
	at com.baidu.hugegraph.core.MultiGraphsTest.testCreateGraphsWithDifferentNameDifferentBackends(MultiGraphsTest.java:301)
[ERROR] testCreateGraphsWithMultiDisksForRocksDB(com.baidu.hugegraph.core.MultiGraphsTest)  Time elapsed: 0.734 s  <<< FAILURE!
java.lang.AssertionError: No exception was thrown(expected com.baidu.hugegraph.backend.BackendException)
	at com.baidu.hugegraph.core.MultiGraphsTest.testCreateGraphsWithMultiDisksForRocksDB(MultiGraphsTest.java:315)
[ERROR] testCreateGraphWithSameNameDifferentBackends(com.baidu.hugegraph.core.MultiGraphsTest)  Time elapsed: 0.003 s  <<< ERROR!
java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class com.baidu.hugegraph.HugeFactory]
	at com.baidu.hugegraph.core.MultiGraphsTest.openGraphWithBackend(MultiGraphsTest.java:373)
	at com.baidu.hugegraph.core.MultiGraphsTest.testCreateGraphWithSameNameDifferentBackends(MultiGraphsTest.java:279)
Caused by: java.lang.reflect.InvocationTargetException
	at com.baidu.hugegraph.core.MultiGraphsTest.openGraphWithBackend(MultiGraphsTest.java:373)
	at com.baidu.hugegraph.core.MultiGraphsTest.testCreateGraphWithSameNameDifferentBackends(MultiGraphsTest.java:279)
Caused by: java.lang.IllegalStateException: Graph name 'graph' has been used by backend 'hbase'
	at com.baidu.hugegraph.core.MultiGraphsTest.openGraphWithBackend(MultiGraphsTest.java:373)
	at com.baidu.hugegraph.core.MultiGraphsTest.testCreateGraphWithSameNameDifferentBackends(MultiGraphsTest.java:279)
[ERROR] testCopySchemaWithMultiGraphs(com.baidu.hugegraph.core.MultiGraphsTest)  Time elapsed: 105.285 s  <<< FAILURE!
java.lang.AssertionError: java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'schema_g1'
	at com.baidu.hugegraph.core.MultiGraphsTest.destoryGraphs(MultiGraphsTest.java:352)
	at com.baidu.hugegraph.core.MultiGraphsTest.testCopySchemaWithMultiGraphs(MultiGraphsTest.java:185)

@codecov
Copy link

codecov bot commented Jul 23, 2020

Codecov Report

Merging #1104 into master will increase coverage by 9.04%.
The diff coverage is 86.66%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1104      +/-   ##
============================================
+ Coverage     60.35%   69.40%   +9.04%     
- Complexity     4709     5625     +916     
============================================
  Files           331      331              
  Lines         26997    27116     +119     
  Branches       3865     3896      +31     
============================================
+ Hits          16295    18819    +2524     
+ Misses         8915     6448    -2467     
- Partials       1787     1849      +62     
Impacted Files Coverage Δ Complexity Δ
...in/java/com/baidu/hugegraph/StandardHugeGraph.java 80.40% <ø> (+1.27%) 108.00 <0.00> (+6.00)
...ain/java/com/baidu/hugegraph/task/TaskManager.java 74.32% <85.18%> (+1.40%) 30.00 <5.00> (+6.00)
...va/com/baidu/hugegraph/task/ServerInfoManager.java 70.05% <100.00%> (+1.09%) 48.00 <2.00> (+4.00)
...om/baidu/hugegraph/backend/query/QueryResults.java 71.15% <0.00%> (-10.48%) 40.00% <0.00%> (+1.00%) ⬇️
...n/java/com/baidu/hugegraph/config/CoreOptions.java 99.04% <0.00%> (+0.02%) 2.00% <0.00%> (ø%)
.../baidu/hugegraph/backend/tx/SchemaTransaction.java 91.54% <0.00%> (+0.49%) 70.00% <0.00%> (+1.00%)
...om/baidu/hugegraph/task/StandardTaskScheduler.java 79.39% <0.00%> (+0.50%) 90.00% <0.00%> (+9.00%)
...n/java/com/baidu/hugegraph/structure/HugeEdge.java 74.74% <0.00%> (+0.51%) 85.00% <0.00%> (+4.00%)
...ava/com/baidu/hugegraph/structure/HugeElement.java 70.52% <0.00%> (+0.52%) 70.00% <0.00%> (+3.00%)
...aidu/hugegraph/backend/store/mysql/MysqlTable.java 81.54% <0.00%> (+0.55%) 96.00% <0.00%> (+1.00%)
... and 134 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 125b2ef...d69b5fa. Read the comment docs.

@houzhizhen houzhizhen merged commit 24335e2 into master Jul 24, 2020
@javeme javeme deleted the schedule-dead-lock branch July 24, 2020 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

server-info-db-worker: Invalid column family specified in write batch
3 participants