Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimize aggregate functions #813

Merged
merged 14 commits into from
Apr 4, 2020
Merged

optimize aggregate functions #813

merged 14 commits into from
Apr 4, 2020

Conversation

javeme
Copy link
Contributor

@javeme javeme commented Jan 5, 2020

implement: #580 #710
Change-Id: I87af5f447c06e7a42c077d3d9eb1a3acde9d3b66

javeme
javeme commented Jan 5, 2020
View reviewed changes
hugegraph-example/src/main/java/com/baidu/hugegraph/example/Example1.java Outdated

System.out.println(">>>>num4="+
graph.traversal().V().hasLabel("person").both().values("age").sum().toList());

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

hugegraph-core/src/main/java/com/baidu/hugegraph/traversal/optimize/HugeCountStepStrategy.java Outdated
}

@Override
// @SuppressWarnings({ "rawtypes", "unchecked" })
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove //

...raph-cassandra/src/main/java/com/baidu/hugegraph/backend/store/cassandra/CassandraTable.java Outdated
Aggregate aggregate = query.aggregateNotNull();
Iterator<Number> results = this.query(query, statement -> {
// Set request timeout to a large value
// TODO read from conf
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO

hugegraph-core/src/main/java/com/baidu/hugegraph/backend/tx/AbstractTransaction.java Outdated
try {
return this.store.queryNumber(squery);
} finally {
this.afterRead(); // TODO: not complete the iteration currently
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove TODO

hugegraph-hbase/src/main/java/com/baidu/hugegraph/backend/store/hbase/HbaseSessions.java Outdated
private AggregationClient aggregationClient() {
Configuration hConfig = this.hbase.getConfiguration();
hConfig = HBaseConfiguration.create(hConfig);
// TODO: read from conf
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO

@javeme
optimize aggregate functions
40dd15e 
Change-Id: I87af5f447c06e7a42c077d3d9eb1a3acde9d3b66
@javeme
put backend dependencies into submodules
3509c12 
Change-Id: If8579e920d950b5573a87f25b1f79f34b7114ca1
@javeme
adapt hbase
2b5cac2 
Change-Id: Iab39d76c764f0b3ed3953be54dcebe51eb8afb7c
@javeme
improve test
2e36b20 
Change-Id: I09dc9aad7458d17e1b7b5b76fa028edbee5501b4
@javeme
adapt to rocksdb
2e4b8ed 
Change-Id: Ie68884edf56c552e1add6158cd82a1c3588bf6eb
@javeme
adapt to mysql
f282425 
Change-Id: Ic83b5c55db8ea089b6adaa9491757a52a9d59cfe
@javeme
adapt to memory backend
307994a 
Change-Id: Icf781d62b09c617c0a2585d8b389617ae4711c63
@javeme
reset conf
3c403d1 
Change-Id: I58c646b4699f1d0653807cb7b9e0a0e2acaaa404
@javeme
fix query count with uncommitted records
5b7b805 
Change-Id: I636ff6c84444f81bdb86eb6efcb040530d85fef8
@javeme
add option aggregation_timeout for cassandra and hbase
f1d0d90 
Change-Id: I0b5a740078251b77d88b8490e9cde965bfe5696c
@javeme
Copy link
Contributor Author

javeme commented Mar 31, 2020

image

Linary
Linary previously approved these changes Apr 3, 2020
View reviewed changes
zhoney
zhoney previously approved these changes Apr 3, 2020
View reviewed changes
@javeme
add travis log for run-api-test
6716f03 
Change-Id: I4bb9bf21126b62bd58f3c5b1f4b1d8513c385612
@javeme javeme dismissed stale reviews from zhoney and Linary via 6716f03 April 3, 2020 14:06
@javeme
fix jersey dependency conflict
cf5cb55 
hadoop depend on jersey 1.9

Change-Id: I7ceab1896b9b72725b1d50327aa66f4396ec8806
@javeme javeme force-pushed the optimize-agg-func branch 5 times, most recently from 30ba06e to 01d72da Compare April 3, 2020 18:54
@javeme
Copy link
Contributor Author

javeme commented Apr 4, 2020 •
edited

java.lang.AbstractMethodError: javax.ws.rs.core.UriBuilder.uri

2020-04-03 18:58:09 9347  [main] [ERROR] com.baidu.hugegraph.dist.HugeGraphServer [] - HugeRestServer start error: 
java.lang.AbstractMethodError: javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
	at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119) ~[javax.ws.rs-api-2.0.1.jar:2.0.1]
	at com.baidu.hugegraph.server.RestServer.start(RestServer.java:56) ~[hugegraph-api-0.11.0.jar:0.50.0.0]
	at com.baidu.hugegraph.server.RestServer.start(RestServer.java:113) ~[hugegraph-api-0.11.0.jar:0.50.0.0]
	at com.baidu.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:58) ~[hugegraph-dist-0.11.0.jar:?]
	at com.baidu.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:55) [hugegraph-dist-0.11.0.jar:?]
	at com.baidu.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:100) [hugegraph-dist-0.11.0.jar:?]

@javeme
fix javax.servlet conflict
6bfd54b 
Change-Id: I3332044f18601076e8c56b66353561ac31793ffd
@javeme
specify specific exclusion names instead of wildcards
1164a78 
Change-Id: If223f62970276e02d4c74877fae811c10afe6459
@codecov
Copy link

codecov bot commented Apr 4, 2020 •
edited

Codecov Report

Merging #813 into master will decrease coverage by 0.06%.
The diff coverage is 72.15%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #813      +/-   ##
============================================
- Coverage     72.30%   72.24%   -0.07%     
- Complexity     4511     4567      +56     
============================================
  Files           287      290       +3     
  Lines         21464    21742     +278     
  Branches       3040     3079      +39     
============================================
+ Hits          15520    15707     +187     
- Misses         4428     4499      +71     
- Partials       1516     1536      +20
Impacted Files Coverage Δ Complexity Δ
...raph/backend/store/cassandra/CassandraMetrics.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...du/hugegraph/backend/cache/CachedBackendStore.java 9.52% <0.00%> (-0.16%) 0.00 <0.00> (ø)
...om/baidu/hugegraph/backend/store/BackendStore.java 87.50% <ø> (ø) 4.00 <0.00> (ø)
...om/baidu/hugegraph/backend/store/BackendTable.java 92.00% <ø> (ø) 12.00 <0.00> (ø)
...u/hugegraph/backend/tx/SchemaIndexTransaction.java 85.36% <ø> (ø) 13.00 <0.00> (ø)
...aidu/hugegraph/traversal/optimize/QueryHolder.java 58.82% <0.00%> (-7.85%) 5.00 <0.00> (ø)
...du/hugegraph/traversal/optimize/TraversalUtil.java 61.03% <0.00%> (-6.11%) 84.00 <0.00> (ø)
...idu/hugegraph/backend/store/hbase/HbaseTables.java 88.57% <ø> (ø) 0.00 <0.00> (ø)
...u/hugegraph/backend/store/mysql/MysqlSessions.java 63.01% <0.00%> (ø) 24.00 <0.00> (ø)
...u/hugegraph/backend/store/hbase/HbaseSessions.java 62.10% <47.69%> (-1.15%) 23.00 <3.00> (+1.00) ⬇️
... and 37 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01f1e77...1164a78. Read the comment docs.

@javeme javeme merged commit 65799ea into master Apr 4, 2020
@javeme javeme deleted the optimize-agg-func branch April 4, 2020 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Linary Linary Linary approved these changes

@zhoney zhoney zhoney approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@javeme @Linary @zhoney