support aggregate property #693

zhoney · 2019-09-20T05:23:59Z

implemented: #691

Change-Id: I340d7be28090718537e1a531147f1e1155a1beb5

codecov · 2019-09-20T06:21:13Z

Codecov Report

Merging #693 into master will increase coverage by 0.04%.
The diff coverage is 85.1%.

@@             Coverage Diff              @@
##             master     #693      +/-   ##
============================================
+ Coverage     74.36%   74.41%   +0.04%     
- Complexity     4097     4143      +46     
============================================
  Files           220      221       +1     
  Lines         17928    18091     +163     
  Branches       2573     2591      +18     
============================================
+ Hits          13332    13462     +130     
- Misses         3237     3253      +16     
- Partials       1359     1376      +17

Impacted Files	Coverage Δ	Complexity Δ
...baidu/hugegraph/backend/store/BackendFeatures.java	`0% <ø> (ø)`	`0 <0> (ø)`	⬇️
.../baidu/hugegraph/backend/query/ConditionQuery.java	`83.48% <ø> (-0.08%)`	`94 <0> (-1)`
...om/baidu/hugegraph/structure/HugeEdgeProperty.java	`58.82% <0%> (ø)`	`7 <1> (+1)`	⬆️
.../baidu/hugegraph/structure/HugeVertexProperty.java	`57.89% <0%> (-8.78%)`	`7 <1> (-1)`
...ackend/store/cassandra/CassandraStoreProvider.java	`100% <100%> (ø)`	`6 <1> (ø)`	⬇️
...u/hugegraph/backend/store/mysql/MysqlFeatures.java	`84.21% <100%> (+0.87%)`	`16 <1> (+1)`	⬆️
...u/hugegraph/backend/serializer/TextSerializer.java	`87.37% <100%> (+0.12%)`	`79 <0> (ø)`	⬇️
...u/hugegraph/backend/store/hbase/HbaseFeatures.java	`84.21% <100%> (+0.87%)`	`16 <1> (+1)`	⬆️
...du/hugegraph/schema/builder/IndexLabelBuilder.java	`87.9% <100%> (+0.09%)`	`85 <0> (ø)`	⬇️
.../hugegraph/backend/serializer/TableSerializer.java	`88.28% <100%> (+0.48%)`	`55 <1> (+1)`	⬆️
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ef07e0...2e34baf. Read the comment docs.

javeme · 2019-09-20T12:25:15Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

            // Update index of vertex(only include props)
            this.indexTx.updateVertexIndex(v, false);
            this.indexTx.updateLabelIndex(v, false);
+            // Update function properties


javeme · 2019-09-20T12:39:04Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

+                    v.removeProperty(key);
+                }
+            }
+            // Add vertex entry without function properties


javeme · 2019-09-20T12:41:34Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/PropertyKey.java

+        Builder calcSum();
+
+        Builder calcOld();
+


javeme · 2019-09-20T12:43:11Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

+                    e.removeProperty(key);
+                }
+            }
+            // Add edge entry of OUT and IN without function properties


javeme · 2019-09-20T12:45:17Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

            // Update index of vertex(only include props)
            this.indexTx.updateVertexIndex(v, false);
            this.indexTx.updateLabelIndex(v, false);
+            // Update function properties
+            Map<Id, HugeProperty<?>> props = v.getAggregateProperties();
+            if (props != null && !props.isEmpty()) {


don't return null

javeme · 2019-09-20T12:45:18Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

            // Update index of edge
            this.indexTx.updateEdgeIndex(e, false);
            this.indexTx.updateLabelIndex(e, false);
+            Map<Id, HugeProperty<?>> props = e.getAggregateProperties();
+            if (props != null && !props.isEmpty()) {


javeme · 2019-09-20T13:40:05Z

hugegraph-test/src/main/java/com/baidu/hugegraph/core/PropertyKeyCoreTest.java

+
+        Assert.assertThrows(NotAllowException.class, () -> {
+            schema.propertyKey("aggregateProperty")
+                  .asUuid().valueSingle().calcMin().create();


rebase https://github.com/hugegraph/hugegraph/pull/673

javeme · 2019-09-24T06:54:34Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/IndexLabelBuilder.java

@@ -319,6 +319,9 @@ private void checkFields(Set<Id> propertyIds) {
                            "Can't build index on undefined property key " +
                            "'%s' for '%s': '%s'", field,
                            this.baseType.readableName(), this.baseValue);
+            E.checkArgument(pkey.aggregateType().isIndexable(),
+                            "The aggregate type %s in not indexable",


is not indexable

javeme · 2019-09-24T06:54:59Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/PropertyKeyBuilder.java

 import com.baidu.hugegraph.type.define.Cardinality;
 import com.baidu.hugegraph.type.define.DataType;
 import com.baidu.hugegraph.util.E;

 public class PropertyKeyBuilder implements PropertyKey.Builder {

+    public static final String TOP_N = "~topN";


javeme · 2019-09-24T06:56:50Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/PropertyKeyBuilder.java

+        if (this.aggregateType.isTopN()) {
+            Object n = this.userdata.get(TOP_N);
+            E.checkArgument(n instanceof Integer && (int) n > 0,
+                            "The top n must > 0, but got: %s", n);


"The top n must be positive int

javeme · 2019-09-24T06:57:29Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/PropertyKeyBuilder.java

+        }
+        if (this.cardinality != Cardinality.SINGLE) {
+            throw new NotAllowException("Not allowed to set aggregate type " +
+                                        "'%s'for property key '%s' with " +


add a space before "for"

javeme · 2019-09-24T06:59:19Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/PropertyKeyBuilder.java

+        if (this.aggregateType.isNumber() &&
+            !this.dataType.isNumber() && !this.dataType.isDate()) {
+            throw new NotAllowException(
+                      "Not allowed to set aggregate type '%s'for " +


javeme · 2019-09-24T07:02:57Z

hugegraph-core/src/main/java/com/baidu/hugegraph/structure/HugeEdgeProperty.java

+               HugeType.PROPERTY : HugeType.AGGR_PROPERTY_E;
+    }
+
+    public Object id() {


move HugeEdgeProperty.id() and HugeVertexProperty.id() to HugeProperty

javeme · 2019-09-24T09:25:56Z

hugegraph-core/src/main/java/com/baidu/hugegraph/schema/builder/PropertyKeyBuilder.java

+            E.checkArgument(n instanceof Integer && (int) n > 0,
+                            "The top n must be positive int, but got: %s", n);
+        }
+        if (this.cardinality != Cardinality.SINGLE) {


the cardinality of topN should be list

javeme · 2019-09-24T12:11:00Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

+    private void checkAggregateProperty(HugeProperty property) {
+        E.checkArgument(!property.isAggregateType() ||
+                        this.store().features().supportsAggregateProperty(),
+                        "The store of '%s' not support aggregate " +


javeme · 2019-09-24T12:13:59Z

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/GraphTransaction.java

+    private void checkAggregateProperty(HugeElement element) {
+        E.checkArgument(element.getAggregateProperties().isEmpty() ||
+                        this.store().features().supportsAggregateProperty(),
+                        "The store of '%s' not support aggregate " +


The %s store does not support aggregate property

implemented: #691 Change-Id: I340d7be28090718537e1a531147f1e1155a1beb5

Change-Id: Ib32b4bb297bce5963b9da56e2f26fd9822ff82ec

Change-Id: Id955abfca780792511193e1dd96937399905799e

Change-Id: I46f7af706c5d56e5960c2e667b570ca31d6bff98

Change-Id: Ib7f0bfa7bbdf2b01893c97c2f3e0f35503c7dc93

Change-Id: I427eb98356bffaae7266e604de91baef2d1e7a15

Change-Id: I795ed575eb30dc5cd87c7bc2dff173d7ca268279

Change-Id: Iaa3c88102580c79cbbfc6f7e168b03acf2bc8574

Change-Id: If6aec687456432c40dd44007f2644c7bfdb7bc9d

Change-Id: I4b6faf76f3ebff59cca2a973435d0f055a9bc93a

javeme · 2019-10-04T15:44:08Z

hugegraph-core/src/main/java/com/baidu/hugegraph/type/define/AggregateType.java

+    SUM(3, "sum"),
+    OLD(4, "old");
+
+    private byte code = 0;


javeme · 2019-10-04T15:44:11Z

hugegraph-core/src/main/java/com/baidu/hugegraph/type/define/AggregateType.java

+    OLD(4, "old");
+
+    private byte code = 0;
+    private String name = null;


Change-Id: I11ea1726ea434cde0a02526ee36b7e1930398265

Change-Id: I4bff43bc4145db4ba63270431129d8637138b19a

Change-Id: I0151cdbf7378b02641aced520cfa6aced45c4e7a

Change-Id: Ib60a1909f7187a06c8c09d331879ae51813cf438

Change-Id: Icc375f2d28c0c126ac9a935bcfbb4d348c4396e0

Change-Id: I0379329c070ce42f2b505236ea03d87e94990c76

Change-Id: I02a1fea3d2fa3d010a57817e047b3604fcf7f5b2

zhoney force-pushed the aggregate branch 2 times, most recently from d0e84bd to 42f7cd2 Compare September 20, 2019 06:21

javeme requested changes Sep 20, 2019

View reviewed changes

zhoney force-pushed the aggregate branch 3 times, most recently from 26cb032 to 166cb20 Compare September 21, 2019 09:53

javeme requested changes Sep 24, 2019

View reviewed changes

zhoney force-pushed the aggregate branch from 54f852b to eabfd5e Compare September 24, 2019 08:55

javeme requested changes Sep 24, 2019

View reviewed changes

javeme reviewed Sep 24, 2019

View reviewed changes

zhoney force-pushed the aggregate branch 2 times, most recently from 2f647d2 to 2a91ac5 Compare September 25, 2019 12:24

zhoney added 8 commits September 27, 2019 11:29

support aggregate property

17d0517

implemented: #691 Change-Id: I340d7be28090718537e1a531147f1e1155a1beb5

improve

6ea071c

Change-Id: Ib32b4bb297bce5963b9da56e2f26fd9822ff82ec

delete topN and improve GTX for aggregate support

67dcab1

Change-Id: Id955abfca780792511193e1dd96937399905799e

improve

d4d71a9

Change-Id: I46f7af706c5d56e5960c2e667b570ca31d6bff98

tiny improve

2a92ed4

Change-Id: Ib7f0bfa7bbdf2b01893c97c2f3e0f35503c7dc93

tiny improve

e48de37

Change-Id: I427eb98356bffaae7266e604de91baef2d1e7a15

add keyLock() support for LockUtil

3413a7b

Change-Id: I795ed575eb30dc5cd87c7bc2dff173d7ca268279

upgrade common to 1.6.15

348e9dd

Change-Id: Iaa3c88102580c79cbbfc6f7e168b03acf2bc8574

zhoney force-pushed the aggregate branch from 8d21c8c to 348e9dd Compare September 27, 2019 03:30

zhoney added 2 commits September 30, 2019 11:38

improve

d6ca567

Change-Id: If6aec687456432c40dd44007f2644c7bfdb7bc9d

tiny improve

2b3ebb4

Change-Id: I4b6faf76f3ebff59cca2a973435d0f055a9bc93a

javeme reviewed Oct 4, 2019

View reviewed changes

zhoney added 5 commits October 9, 2019 10:59

tiny improve

07e5bdd

Change-Id: I11ea1726ea434cde0a02526ee36b7e1930398265

add unittest for LockUtil.lockKey()

ba24bd6

Change-Id: I4bff43bc4145db4ba63270431129d8637138b19a

tiny improve

8d324e3

Change-Id: I0151cdbf7378b02641aced520cfa6aced45c4e7a

add keylock2 of common

738f5f6

Change-Id: Ib60a1909f7187a06c8c09d331879ae51813cf438

add unittest for keylock2

78251a3

Change-Id: Icc375f2d28c0c126ac9a935bcfbb4d348c4396e0

zhoney added 2 commits October 10, 2019 18:38

add test for "update if" override "update" for aggregate property

b6ce7e9

Change-Id: I0379329c070ce42f2b505236ea03d87e94990c76

change keylock2 to rowlock

2e34baf

Change-Id: I02a1fea3d2fa3d010a57817e047b3604fcf7f5b2

Linary approved these changes Oct 12, 2019

View reviewed changes

javeme approved these changes Oct 12, 2019

View reviewed changes

javeme merged commit 00faffd into master Oct 12, 2019

javeme deleted the aggregate branch October 12, 2019 09:32

gwny mentioned this pull request Nov 20, 2019

属性支持aggregate问题 #755

Closed

zhoney mentioned this pull request Jul 2, 2021

support aggregate property #691

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support aggregate property #693

support aggregate property #693

zhoney commented Sep 20, 2019

codecov bot commented Sep 20, 2019 •

edited

javeme Sep 20, 2019

javeme Sep 20, 2019

javeme Sep 20, 2019

javeme Sep 20, 2019

javeme Sep 20, 2019

javeme Sep 20, 2019

javeme Sep 20, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Sep 24, 2019

javeme Oct 4, 2019

javeme Oct 4, 2019

support aggregate property #693

support aggregate property #693

Conversation

zhoney commented Sep 20, 2019

codecov bot commented Sep 20, 2019 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Sep 20, 2019 •

edited