support shard-index and vertex + sortkey prefix query #574

zhoney · 2019-06-18T07:53:22Z

implemented: #270 and #398

Change-Id: I444b67b7f57a18fd32c29d54d776cf411ccd847a

codecov · 2019-06-18T08:06:08Z

Codecov Report

Merging #574 into master will decrease coverage by 0.35%.
The diff coverage is 87.72%.

@@             Coverage Diff              @@
##             master     #574      +/-   ##
============================================
- Coverage     71.94%   71.58%   -0.36%     
- Complexity     3640     3686      +46     
============================================
  Files           219      218       -1     
  Lines         16882    17289     +407     
  Branches       2399     2435      +36     
============================================
+ Hits          12145    12376     +231     
- Misses         3475     3631     +156     
- Partials       1262     1282      +20

Impacted Files	Coverage Δ	Complexity Δ
.../hugegraph/backend/serializer/TableSerializer.java	`81.34% <ø> (ø)`	`52 <0> (ø)`	⬇️
.../java/com/baidu/hugegraph/version/CoreVersion.java	`80% <ø> (ø)`	`2 <0> (ø)`	⬇️
...java/com/baidu/hugegraph/type/define/DataType.java	`87.32% <0%> (-2.54%)`	`36 <0> (ø)`
...raph/backend/store/rocksdbsst/RocksDBSstStore.java	`0% <0%> (ø)`	`0 <0> (ø)`	⬇️
...in/java/com/baidu/hugegraph/schema/IndexLabel.java	`69.11% <0%> (ø)`	`25 <2> (ø)`	⬇️
...hugegraph/backend/store/rocksdb/RocksDBTables.java	`55.85% <100%> (-0.47%)`	`0 <0> (ø)`
...idu/hugegraph/backend/store/hbase/HbaseTables.java	`87.5% <100%> (+2.5%)`	`0 <0> (ø)`	⬇️
...kend/store/postgresql/PostgresqlStoreProvider.java	`91.89% <100%> (+0.98%)`	`5 <1> (ø)`	⬇️
...aidu/hugegraph/backend/store/hbase/HbaseStore.java	`78.04% <100%> (+1.12%)`	`29 <0> (ø)`	⬇️
...egraph/backend/store/cassandra/CassandraStore.java	`69.02% <100%> (+0.46%)`	`48 <0> (ø)`	⬇️
... and 47 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 674a5e7...91dcacb. Read the comment docs.

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinaryBackendEntry.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

...raph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/memory/InMemoryDBTables.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/IndexLabelBuilder.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/IndexLabelBuilder.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/IndexLabelBuilder.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

javeme · 2019-07-12T03:35:20Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

+            return false;
+        }
+        List<Id> prefix = sortKeys.subList(0, queryKeys.size());
+        return queryKeys.containsAll(prefix);


rebase https://github.com/hugegraph/hugegraph/pull/512

can also copy from #512 if can't rebase

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/type/HugeType.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

...aph-cassandra/src/main/java/com/baidu/hugegraph/backend/store/cassandra/CassandraTables.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/type/HugeType.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

implemented: #270 and #398 Change-Id: I444b67b7f57a18fd32c29d54d776cf411ccd847a

Change-Id: I3b007538ec5279480f85f42cb126e13c3028dfde

Change-Id: I1e7a7fcf06851e306b723e05c4aad6fe4664c3ef

Change-Id: Ied584d80d05264ec11bb9d03c30bc73e9793ac97

Change-Id: I86ad27ab094660cdf2b4dff0cff31fc36e79683d

Change-Id: I829aa16a0c89af9d16de4b2d18b8afc0f886178c

Change-Id: Ibde8426fee990e257b3f911815b2adbd25526d8c

Change-Id: I64df619d14947fe8441c8eb0ab8cb9f4b302607d

and support prefix + range query Change-Id: Ia60d2422039c7376180603e6b81c40a785e320d7

Change-Id: I03e1421b599e4b57756948894f8b6253b69f25b6

…inarySerializer Change-Id: Ib5f4e64a5e68727b5765076d5458ccc8dd3443c0

Change-Id: I6368833708b7524ad37fc30cab230740c23cd17a

Change-Id: I704a760827a0818457b9afdde102f2b297b68f32

Change-Id: Ibd9f83d8d42bdce4d9b925b362d9af74a081df1e

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinaryBackendEntry.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

Change-Id: Ia8ca6815e396e839dc41d5daadee9db65f50f40e

hugegraph-core/src/main/java/com/baidu/hugegraph/structure/HugeIndex.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdb/RocksDBTables.java

hugegraph-hbase/src/main/java/com/baidu/hugegraph/backend/store/hbase/HbaseTables.java

Change-Id: I409390a5753bdd2052ea4dd12218fde818ac1ad1

Change-Id: Id2b46cfbff907b26862e3225fcd51c556a22b9bf

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/IndexLabel.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/IndexLabelBuilder.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

Change-Id: I93f15c5df19a8fd60b66d73e0e5e40aa83e928e6

hugegraph-core/src/main/java/com/baidu/hugegraph/structure/HugeIndex.java

hugegraph-test/src/main/java/com/baidu/hugegraph/core/IndexLabelCoreTest.java

javeme · 2019-08-02T03:02:18Z

hugegraph-test/src/main/java/com/baidu/hugegraph/core/VertexCoreTest.java

+        graph().addVertex(T.label, "number", "id", 9,
+                          "long", Long.MAX_VALUE - 1);
+        graph().addVertex(T.label, "number", "id", 10,
+                          "long", Long.MIN_VALUE + 1);


add this kind case for float/double in edge test

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java

Change-Id: Ieb23f49144a16e5f5bec097381144f9bb1ccb0c0

hugegraph-test/src/main/java/com/baidu/hugegraph/core/IndexLabelCoreTest.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

javeme · 2019-08-02T05:59:52Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

+            return false;
+        }
+        List<Id> prefix = sortKeys.subList(0, queryKeys.size());
+        return queryKeys.containsAll(prefix);


can also copy from #512 if can't rebase

Change-Id: I077d193d17fd136e7b759eb25b2d10b51c009b8b

hugegraph-core/src/main/java/com/baidu/hugegraph/type/define/IndexType.java

hugegraph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdb/RocksDBTables.java

...raph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdbsst/RocksDBSstStore.java

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/id/IdUtil.java

Change-Id: Ie412a104f66781da5069a2d078b6758b2ec7a912

Change-Id: I69b80144bfd31939938404f2ff15edc5bdbd9998

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/TableSerializer.java

hugegraph-core/src/main/java/com/baidu/hugegraph/structure/HugeIndex.java

Change-Id: I9680de412db11093e199b32b344e40696ac5daf8

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java

Change-Id: Icca9bbc060741d7d3c4c090bc0bc5c664a45a252

javeme requested changes Jun 18, 2019

View reviewed changes

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java Outdated Show resolved Hide resolved

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java Outdated Show resolved Hide resolved

zhoney force-pushed the shard-index branch 3 times, most recently from 95b5578 to dc96d65 Compare June 26, 2019 08:42

javeme reviewed Jun 26, 2019

View reviewed changes

javeme requested changes Jun 26, 2019

View reviewed changes

...raph-rocksdb/src/main/java/com/baidu/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java Outdated Show resolved Hide resolved

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java Outdated Show resolved Hide resolved

javeme requested changes Jun 29, 2019

View reviewed changes

javeme requested changes Jul 3, 2019

View reviewed changes

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphIndexTransaction.java Outdated Show resolved Hide resolved

javeme requested changes Jul 4, 2019

View reviewed changes

javeme requested changes Jul 11, 2019

View reviewed changes

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java Outdated Show resolved Hide resolved

javeme requested changes Jul 12, 2019

View reviewed changes

javeme requested changes Jul 17, 2019

View reviewed changes

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java Outdated Show resolved Hide resolved

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BytesBuffer.java Outdated Show resolved Hide resolved

javeme requested changes Jul 22, 2019

View reviewed changes

zhoney force-pushed the shard-index branch from b55b6d9 to f27b45d Compare July 22, 2019 07:01

javeme requested changes Jul 22, 2019

View reviewed changes

javeme requested changes Jul 24, 2019

View reviewed changes

zhoney force-pushed the shard-index branch from 44c2646 to b3ad5ed Compare July 29, 2019 07:24

zhoney added 9 commits July 31, 2019 11:08

support shard-index and vertex + sortkey prefix query

2ba3181

implemented: #270 and #398 Change-Id: I444b67b7f57a18fd32c29d54d776cf411ccd847a

support shard-index for rocksdb

6f5100a

Change-Id: I3b007538ec5279480f85f42cb126e13c3028dfde

upgrade hugegraph-common to 1.6.7

0c4e72d

Change-Id: I1e7a7fcf06851e306b723e05c4aad6fe4664c3ef

improve

8a16c0d

Change-Id: Ied584d80d05264ec11bb9d03c30bc73e9793ac97

improve

00b2c18

Change-Id: I86ad27ab094660cdf2b4dff0cff31fc36e79683d

add 0xff suffix after field values of index

9cb73c2

Change-Id: I829aa16a0c89af9d16de4b2d18b8afc0f886178c

check string index not contains 0xff

cbf25f1

Change-Id: Ibde8426fee990e257b3f911815b2adbd25526d8c

improve

bb0dae7

Change-Id: I64df619d14947fe8441c8eb0ab8cb9f4b302607d

support arbitrary fields of shard-index and sortkeys

3470e5c

and support prefix + range query Change-Id: Ia60d2422039c7376180603e6b81c40a785e320d7

zhoney added 4 commits July 31, 2019 11:31

move escape and unescape from StringUtil to IdUtil

1bda248

Change-Id: I03e1421b599e4b57756948894f8b6253b69f25b6

cassandra range4&8 use decimal still and merge shard with range for b…

601b882

…inarySerializer Change-Id: Ib5f4e64a5e68727b5765076d5458ccc8dd3443c0

split range table to rangeInt, rangeFloat, rangeLong and rangeDouble

2e70251

Change-Id: I6368833708b7524ad37fc30cab230740c23cd17a

improve

dd45c9f

Change-Id: I704a760827a0818457b9afdde102f2b297b68f32

zhoney force-pushed the shard-index branch 2 times, most recently from 5db717a to 19e6b80 Compare July 31, 2019 03:38

rebase and upgrade to 0.10.2

16e38b3

Change-Id: Ibd9f83d8d42bdce4d9b925b362d9af74a081df1e

zhoney force-pushed the shard-index branch from 19e6b80 to 16e38b3 Compare July 31, 2019 03:47

javeme requested changes Jul 31, 2019

View reviewed changes

improve

d04ef67

Change-Id: Ia8ca6815e396e839dc41d5daadee9db65f50f40e

javeme requested changes Aug 1, 2019

View reviewed changes

zhoney added 2 commits August 1, 2019 12:21

extract range conditon to RangeConditions

ea5aeaf

Change-Id: I409390a5753bdd2052ea4dd12218fde818ac1ad1

unify table names using HugeType.string()

e11aaec

Change-Id: Id2b46cfbff907b26862e3225fcd51c556a22b9bf

javeme requested changes Aug 1, 2019

View reviewed changes

improve

325bc08

Change-Id: I93f15c5df19a8fd60b66d73e0e5e40aa83e928e6

javeme requested changes Aug 2, 2019

View reviewed changes

improve

a182a0b

Change-Id: Ieb23f49144a16e5f5bec097381144f9bb1ccb0c0

javeme requested changes Aug 2, 2019

View reviewed changes

improve

057f473

Change-Id: I077d193d17fd136e7b759eb25b2d10b51c009b8b

Linary requested changes Aug 2, 2019

View reviewed changes

zhoney added 2 commits August 2, 2019 16:20

improve

78fa25b

Change-Id: Ie412a104f66781da5069a2d078b6758b2ec7a912

improve

61ee633

Change-Id: I69b80144bfd31939938404f2ff15edc5bdbd9998

javeme requested changes Aug 2, 2019

View reviewed changes

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/TableSerializer.java Outdated Show resolved Hide resolved

hugegraph-core/src/main/java/com/baidu/hugegraph/structure/HugeIndex.java Show resolved Hide resolved

tiny improve

0c4d72b

Change-Id: I9680de412db11093e199b32b344e40696ac5daf8

javeme reviewed Aug 5, 2019

View reviewed changes

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/serializer/BinarySerializer.java Outdated Show resolved Hide resolved

tiny improve

91dcacb

Change-Id: Icca9bbc060741d7d3c4c090bc0bc5c664a45a252

javeme approved these changes Aug 6, 2019

View reviewed changes

Linary approved these changes Aug 6, 2019

View reviewed changes

Linary merged commit 0e66c64 into master Aug 6, 2019

Linary deleted the shard-index branch August 6, 2019 03:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support shard-index and vertex + sortkey prefix query #574

support shard-index and vertex + sortkey prefix query #574

zhoney commented Jun 18, 2019

codecov bot commented Jun 18, 2019 •

edited

javeme Jul 12, 2019

javeme Aug 2, 2019

javeme Aug 2, 2019

javeme Aug 2, 2019

support shard-index and vertex + sortkey prefix query #574

support shard-index and vertex + sortkey prefix query #574

Conversation

zhoney commented Jun 18, 2019

codecov bot commented Jun 18, 2019 • edited

Codecov Report

javeme Jul 12, 2019

Choose a reason for hiding this comment

javeme Aug 2, 2019

Choose a reason for hiding this comment

javeme Aug 2, 2019

Choose a reason for hiding this comment

javeme Aug 2, 2019

Choose a reason for hiding this comment

codecov bot commented Jun 18, 2019 •

edited