Support to create graph dynamically #1065

Linary · 2020-06-30T14:27:46Z

Implement #165

Change-Id: I03d3e606d6292acec46a7725094c05fe84a6c9cd

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

javeme · 2020-07-01T06:32:31Z

hugegraph-api/src/main/java/com/baidu/hugegraph/server/RestServer.java

@@ -69,6 +78,17 @@ public void start() throws IOException {
        this.calcMaxWriteThreads();
    }

+    private void fillGraphsOption(String graphsDir) {


move to graph manager

hugegraph-api/src/main/java/com/baidu/hugegraph/server/RestServer.java

hugegraph-api/src/main/java/com/baidu/hugegraph/config/ServerOptions.java

javeme · 2020-07-01T06:42:29Z

hugegraph-dist/src/main/java/com/baidu/hugegraph/cmd/InitStore.java

 import com.baidu.hugegraph.util.E;
 import com.baidu.hugegraph.util.Log;

 public class InitStore {

    private static final Logger LOG = Log.logger(InitStore.class);

-    private static final String GRAPHS = ServerOptions.GRAPHS.name();
+    private static final String GRAPHS = "graphs";


still keep it

move to ConfigUtil

hugegraph-api/src/main/java/com/baidu/hugegraph/api/profile/GraphsAPI.java

javeme · 2020-07-01T07:00:33Z

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

@@ -88,6 +96,17 @@ public void loadGraphs(final Map<String, String> graphConfs) {
        }
    }

+    public HugeGraph createGraph(HugeConfig config) {


ensure auth permission: write graph

hugegraph-api/src/main/java/com/baidu/hugegraph/auth/ContextGremlinServer.java

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

hugegraph-api/src/main/java/com/baidu/hugegraph/api/profile/GraphsAPI.java

javeme · 2020-07-02T04:14:34Z

hugegraph-api/src/main/java/com/baidu/hugegraph/api/profile/GraphsAPI.java

+    @Timed
+    @Path("{name}")
+    @Produces(APPLICATION_JSON_WITH_CHARSET)
+    @RolesAllowed({"admin"})


hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

hugegraph-dist/src/assembly/static/conf/gremlin-server.yaml

hugegraph-dist/src/assembly/static/conf/rest-server.properties

hugegraph-api/src/main/java/com/baidu/hugegraph/api/profile/GraphsAPI.java

hugegraph-api/src/main/java/com/baidu/hugegraph/auth/ContextGremlinServer.java

javeme · 2020-07-06T03:24:46Z

hugegraph-dist/src/main/java/com/baidu/hugegraph/dist/HugeGraphServer.java


        HugeGraphServer server = new HugeGraphServer(args[0], args[1]);
        Runtime.getRuntime().addShutdownHook(new Thread(() -> {
            LOG.info("HugeGraphServer stopping");
            server.stop();
-            LOG.info("HugeGraphServer stopped");


sure to remove it?

not sure, just feel the logs are a bit redundant

hugegraph-dist/src/main/java/com/baidu/hugegraph/dist/HugeGremlinServer.java

javeme · 2020-07-16T05:15:33Z

api test error:

# Set backend and serializer
sed -i "s/backend=.*/backend=$BACKEND/" $CONF
sed: can't read hugegraph-0.11.1/conf/hugegraph.properties: No such file or directory
cat: hugegraph-0.11.1/logs/hugegraph-server.log: No such file or directory
The command "$TRAVIS_DIR/run-api-test.sh" exited with 1.
312.45s$ $TRAVIS_DIR/run-unit-test.sh

javeme · 2020-07-16T08:48:35Z

cassandra:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20:test (core-test) on project hugegraph-test: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/travis/build/hugegraph/hugegraph/hugegraph-test/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hugegraph-test
The command "mvn test -P core-test,$BACKEND" exited with 1.

postgre:

[ERROR] Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.4:dump (pull-test-data) on project hugegraph-test: Unable to dump coverage data: Connection refused (Connection refused) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
The command "$TRAVIS_DIR/run-api-test.sh" exited with 1.

codecov · 2020-08-11T09:31:20Z

Codecov Report

Merging #1065 (24972bb) into master (40a85ed) will decrease coverage by 0.14%.
The diff coverage is 30.43%.

@@             Coverage Diff              @@
##             master    #1065      +/-   ##
============================================
- Coverage     66.77%   66.62%   -0.15%     
- Complexity     7063     7092      +29     
============================================
  Files           420      423       +3     
  Lines         34726    34953     +227     
  Branches       4822     4844      +22     
============================================
+ Hits          23187    23289     +102     
- Misses         9196     9307     +111     
- Partials       2343     2357      +14

Impacted Files	Coverage Δ
...src/main/java/com/baidu/hugegraph/HugeFactory.java	`70.37% <0.00%> (-5.63%)`	⬇️
...java/com/baidu/hugegraph/type/define/NodeRole.java	`68.75% <ø> (ø)`
...src/main/java/com/baidu/hugegraph/util/Events.java	`0.00% <ø> (ø)`
...c/main/java/com/baidu/hugegraph/cmd/InitStore.java	`0.00% <0.00%> (ø)`
...ava/com/baidu/hugegraph/api/profile/GraphsAPI.java	`10.44% <7.69%> (-0.27%)`	⬇️
...in/java/com/baidu/hugegraph/core/GraphManager.java	`37.16% <12.08%> (-14.56%)`	⬇️
...com/baidu/hugegraph/auth/ContextGremlinServer.java	`32.25% <21.05%> (-21.59%)`	⬇️
...main/java/com/baidu/hugegraph/util/ConfigUtil.java	`48.78% <48.78%> (ø)`
.../com/baidu/hugegraph/server/ApplicationConfig.java	`91.48% <71.42%> (-1.99%)`	⬇️
...va/com/baidu/hugegraph/dist/HugeGremlinServer.java	`52.38% <75.00%> (+20.95%)`	⬆️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 40a85ed...24972bb. Read the comment docs.

hugegraph-dist/src/assembly/static/conf/gremlin-server.yaml

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

javeme · 2020-09-30T09:53:21Z

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

+        // but it's not easy to check here
+        String backend = config.get(CoreOptions.BACKEND);
+        if (backend.equalsIgnoreCase("rocksdb")) {
+            // TODO: should check data path...


seem same data path is ok

same data path is not ok

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

dyhyao6 · 2020-10-24T04:19:59Z

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

+                        "The graph name '%s' has existed", name);
+
+        PropertiesConfiguration propConfig = this.buildConfig(configText);
+        HugeConfig config = new HugeConfig(propConfig, false);


Cannot resolve constructor 'HugeConfig(org.apache.commons.configuration.PropertiesConfiguration, boolean);
are you sure? @ @javeme @Linary

This constructor was defined in hugegraph-common 1.7.10, it's not yet published to the maven central repository.

dyhyao6 · 2020-11-05T02:34:24Z

能不能加快实现图的热更新，thanks @Linary @javeme

javeme · 2020-11-24T03:46:24Z

hugegraph-dist/src/assembly/static/bin/start-hugegraph.sh

@@ -57,8 +57,8 @@ check_port "$REST_SERVER_URL"

 echo "Starting HugeGraphServer..."

-"$BIN"/hugegraph-server.sh "$CONF"/gremlin-server.yaml "$CONF"/rest-server.properties \
-"$OPEN_SECURITY_CHECK" "$USER_OPTION" "$GC_OPTION" >>"$LOGS/hugegraph-server.log" 2>&1 &
+${BIN}/hugegraph-server.sh ${CONF}/gremlin-server.yaml ${CONF}/rest-server.properties \


mkdir -p ${LOGS}

github-actions · 2021-07-22T21:14:05Z

Due to the lack of activity, the current pr is marked as stale and will be closed after 180 days, any update will remove the stale label

Implement #165 Change-Id: I03d3e606d6292acec46a7725094c05fe84a6c9cd

Change-Id: I5626be6a0c56074c51339283bf6f6e40914ae56b

Change-Id: I3816cdd3f2ab41f8976256956dd7dbb5fa6bab89

Change-Id: I276bf797a7cd601aaf6c9de1a5148cf05fba9b7d

Change-Id: I0a7f08454be5313478ecc0596731a6bc5f7dec98

Change-Id: I36e8815dd7d1ecac5615dd6a8638ff5ad13414d2

Change-Id: I57ca9ed3db38c77189714840de24b6c5e5e78ad1

Change-Id: Iedc438f6dee08e48dce2868e69555aaf0c058401

Change-Id: I08898184d6bc0323876dfb4a7398b7fafd43f8db

Change-Id: I37a92977c33332e5e8ab9f2b502c2283aa6b0343

javeme · 2021-12-22T12:40:15Z

hugegraph-api/src/main/java/com/baidu/hugegraph/api/profile/GraphsAPI.java

+    @Consumes(MediaType.TEXT_PLAIN)
+    @Produces(APPLICATION_JSON_WITH_CHARSET)
+    @RolesAllowed({"admin", "$owner=$name"})
+    public Object manage(@Context GraphManager manager,


add delete method and move clone to create

Change-Id: I4c0220a2f22163037449a6cb829756ad78f7b942

Change-Id: I3322cd9be94b9137ed96e5d8f9211af7f54f926f

javeme · 2021-12-23T15:30:20Z

hugegraph-api/src/main/java/com/baidu/hugegraph/api/profile/GraphsAPI.java

-        LOG.debug("Create graph {} with config options '{}'", name, configText);
-        HugeGraph graph = manager.createGraph(name, configText);
+        HugeGraph graph;
+        if (clone != null && clone.isEmpty()) {


should be "!clone.isEmpty()"?

Change-Id: I87dd6412b280f3c04ba5a405bb4d9013270adf3d

Change-Id: I4a9b25d4a2a5b0f722cb0f46b5a93f28e4ccccb2

javeme · 2021-12-24T07:01:47Z

hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java

+        return graph;
+    }
+
+    private PropertiesConfiguration buildConfig(String configText) {


check configText not null? and seems empty string is ok

Change-Id: If328e38979f7ee65b745fe52fcbf7c9c76b24df1

imbajin

test in client side

javeme requested changes Jul 1, 2020

View reviewed changes

javeme reviewed Jul 1, 2020

View reviewed changes

Linary force-pushed the dynamic-add-graph branch from b734d9d to c737e2c Compare July 1, 2020 14:49

javeme reviewed Jul 2, 2020

View reviewed changes

hugegraph-dist/src/assembly/static/conf/gremlin-server.yaml Outdated Show resolved Hide resolved

hugegraph-dist/src/assembly/static/conf/rest-server.properties Outdated Show resolved Hide resolved

Linary force-pushed the dynamic-add-graph branch 5 times, most recently from eb6230e to 80ad81d Compare July 2, 2020 10:16

javeme reviewed Jul 6, 2020

View reviewed changes

Linary force-pushed the dynamic-add-graph branch from 2d8c4fe to b2f2a5c Compare July 15, 2020 08:06

Linary force-pushed the dynamic-add-graph branch from 233997f to 40647c1 Compare July 16, 2020 07:28

Linary force-pushed the dynamic-add-graph branch from 40647c1 to 38bc15b Compare August 11, 2020 09:10

Linary force-pushed the dynamic-add-graph branch from 38bc15b to 46b17b0 Compare September 30, 2020 06:22

javeme requested changes Sep 30, 2020

View reviewed changes

javeme mentioned this pull request Oct 23, 2020

目前是否支持动态新增图，而不用在配置文件里配置，初始化，重启等步骤，thx #1222

Closed

dyhyao6 reviewed Oct 24, 2020

View reviewed changes

Linary mentioned this pull request Oct 28, 2020

Hugegraph 能否支持热更新,在server中新加一个图的情况下,不用重启server #165

Open

Linary force-pushed the dynamic-add-graph branch 2 times, most recently from 289ec6c to 6f5b3c0 Compare October 30, 2020 07:35

javeme reviewed Nov 24, 2020

View reviewed changes

Linary force-pushed the dynamic-add-graph branch from 6f5b3c0 to 221aaad Compare November 26, 2020 08:47

github-actions bot added the inactive label Jul 22, 2021

imbajin removed the inactive label Jul 26, 2021

github-actions bot removed the inactive label Oct 9, 2021

Linary and others added 9 commits December 22, 2021 10:58

Support to create graph dynamically

df7b1bc

Implement #165 Change-Id: I03d3e606d6292acec46a7725094c05fe84a6c9cd

merge gremlin-server graphs and specified graphs

4203ed1

Change-Id: I5626be6a0c56074c51339283bf6f6e40914ae56b

Just use graphs directory as the graph source

69bd2ec

Change-Id: I3816cdd3f2ab41f8976256956dd7dbb5fa6bab89

tiny improve

8d5151c

Change-Id: I276bf797a7cd601aaf6c9de1a5148cf05fba9b7d

modify start-server script

c6bc2f5

Change-Id: I0a7f08454be5313478ecc0596731a6bc5f7dec98

use text/plain media type

9ec05ad

Change-Id: I36e8815dd7d1ecac5615dd6a8638ff5ad13414d2

some improve

e886808

Change-Id: I57ca9ed3db38c77189714840de24b6c5e5e78ad1

improve after rebase master

e350a1e

Change-Id: Iedc438f6dee08e48dce2868e69555aaf0c058401

use notify() instead of call() of Eventhub for graph add/remove

3507850

Change-Id: I08898184d6bc0323876dfb4a7398b7fafd43f8db

zhoney force-pushed the dynamic-add-graph branch from 35aea43 to 3507850 Compare December 22, 2021 03:05

fix ci

86df5d2

Change-Id: I37a92977c33332e5e8ab9f2b502c2283aa6b0343

javeme previously approved these changes Dec 22, 2021

View reviewed changes

javeme reviewed Dec 22, 2021

View reviewed changes

merge clone graph into create graph

c022bbe

Change-Id: I4c0220a2f22163037449a6cb829756ad78f7b942

zhoney dismissed javeme’s stale review via c022bbe December 23, 2021 13:12

improve

ed1462a

Change-Id: I3322cd9be94b9137ed96e5d8f9211af7f54f926f

javeme reviewed Dec 23, 2021

View reviewed changes

zhoney added 2 commits December 24, 2021 10:06

improve

d214d32

Change-Id: I87dd6412b280f3c04ba5a405bb4d9013270adf3d

improve

4858166

Change-Id: I4a9b25d4a2a5b0f722cb0f46b5a93f28e4ccccb2

javeme previously approved these changes Dec 24, 2021

View reviewed changes

javeme reviewed Dec 24, 2021

View reviewed changes

improve

24972bb

Change-Id: If328e38979f7ee65b745fe52fcbf7c9c76b24df1

zhoney dismissed javeme’s stale review via 24972bb December 24, 2021 07:14

javeme approved these changes Dec 24, 2021

View reviewed changes

imbajin approved these changes Dec 24, 2021

View reviewed changes

javeme merged commit d213166 into master Dec 24, 2021

javeme deleted the dynamic-add-graph branch December 24, 2021 11:25

simon824 mentioned this pull request Apr 22, 2022

如何动态创建图 #1825

Closed

1 task

imbajin mentioned this pull request May 14, 2023

[Question] how to use hugegraph-tools to create new graph instance apache/incubator-hugegraph-toolchain#461

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support to create graph dynamically #1065

Support to create graph dynamically #1065

Linary commented Jun 30, 2020

javeme Jul 1, 2020

javeme Jul 1, 2020

Linary Jul 1, 2020

javeme Jul 1, 2020

javeme Jul 2, 2020

javeme Jul 6, 2020

Linary Jul 6, 2020

javeme commented Jul 16, 2020

javeme commented Jul 16, 2020

codecov bot commented Aug 11, 2020 •

edited

javeme Sep 30, 2020

Linary Oct 30, 2020

dyhyao6 Oct 24, 2020

Linary Oct 28, 2020

dyhyao6 commented Nov 5, 2020

javeme Nov 24, 2020

github-actions bot commented Jul 22, 2021

javeme Dec 22, 2021

javeme Dec 23, 2021

javeme Dec 24, 2021

imbajin left a comment

Support to create graph dynamically #1065

Support to create graph dynamically #1065

Conversation

Linary commented Jun 30, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

javeme commented Jul 16, 2020

javeme commented Jul 16, 2020

codecov bot commented Aug 11, 2020 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dyhyao6 commented Nov 5, 2020

Choose a reason for hiding this comment

github-actions bot commented Jul 22, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imbajin left a comment

Choose a reason for hiding this comment

codecov bot commented Aug 11, 2020 •

edited