Skip to content

Release v0.18.0
Compare
Choose a tag to compare
@lidongze0629 lidongze0629 released this 02 Dec 02:28
· 962 commits to main since this release
v0.18.0
6775658
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.18.0 release. This release contains many important features and enhancements to Graph Interactive Engine (GIE), including introducing a new strategy for pattern-matching queries and supporting canceling running queries. The Graph Learning Engine (GLE) now supports PyTorch and is also compatible with PyG. In addition, we take a first step towards modularized deployment for different components of GraphScope in Kubernetes.

We highlight the following improvements included in this release:

1. Enhancements for GIE

  • A new execution strategy based on worst-case optimal join is introduced to GIE engine, potentially improving the performance of match step by orders of magnitude.
  • The query can be canceled after its execution exceeds a pre-given overtime parameter (10min by default).
  • GIE engine supports a failover mechanism: if an executor pod fails, but the data is not missing, it can be restarted by k8s. Existing queries cannot recover, but the engine can serve the following.

2. Enhancements for GAE

  • Add more variants of WCC algorithm.
  • Supports local vertex map to make it could scale to larger graphs given more workers.

3. Enhancements for GLE

  • Add support for PyTorch and PyG
  • Add heterogeneous graph support for subgraph-based GNN, add HeteroSubGraph and HeteroConv, bipartite GraphSAGE and UltraGCN.
  • Add edge feature support in both EgoGraph and SubGraph.
  • Add recommendation metrics: Recall, NDCG and HitRate.
  • Add hiactor-based graph engine.

4. Standalone deployment

  • Supports a version of GIE standalone deployment in Kubernetes.
  • Redesigned docker files from the bottom to the top, making it more clear and more concise.

5. GAE Java Enhancement

  • Introduce @GrapeSkip for better user experience in template method overloading.
  • Speedup Java App runimte codegen.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.18.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.18.0

Commits

  • [6775658]: Push jupyter image nightly with the latest client (#2266) (DongZe Li)
  • [df9f6eb]: Fix nightly CI (#2262) (Siyuan Zhang)
  • [4f17777]: Fixes CI failure both on Linux and MacOS (#2244) (DongZe Li)
  • [07efc74]: [GIE/engine] Bug fix; (#2250) (bmmcq)
  • [5ea5b87]: [GIE] Support parallel scan on ExpStore (#2253) (BingqingLyu)
  • [432e65c]: [GIE] Make the version of GIE compiler consistent with the default value in interactive engine pom (#2249) (shirly121)
  • [2b7cf00]: [GIE POM] Unify GIE version by the global flag 'revision' (#2243) (shirly121)
  • [71d61e3]: Reorganize for Makefiles and dockerfiles (#2246) (Siyuan Zhang)
  • [84f449c]: [GIE Runtime] Debug runtime op info on in a more clearly way (#2226) (BingqingLyu)
  • [f77627d]: Update the learning model to align with the latest graphlearn (#2235) (Tao He)
  • [1c4c1cd]: [GIE/Store] Reduce memory footprint in exp_store (#2245) (Longbin Lai)
  • [e375f8d]: [GIE/Exp-Store] Optimize memory usage in loading graph data into MutTopo. (#2242) (Longbin Lai)
  • [8844e90]: Fixes release graphscope-store image (#2241) (Siyuan Zhang)
  • [428c785]: [GIE/Engine]: add direct executor implementation; (#2234) (bmmcq)
  • [e071ce4]: add docs revision preview in CI (#2237) (Jingbo Xu)
  • [f524550]: Enable cpplint in CI (#2233) (Tao He)
  • [330fe14]: Makes GAE compatible with latest arrow release (#2232) (Tao He)
  • [ed86040]: Revise docs. (#2231) (Jingbo Xu)
  • [b4d7038]: Fixes compilation error of graphx runner (#2228) (Siyuan Zhang)
  • [a11b453]: Reorganize Dockerfile according to specification of standalone deployment (#2227) (Siyuan Zhang)
  • [dcaa38a]: [GIE/GLogue] rename Catalogue as GLogue (#2224) (BingqingLyu)
  • [aed1f13]: [GIE/IR] Introduce a new strategy for pattern matching (#2159) (BingqingLyu)
  • [92f7da3]: Add two other version of wcc for compatiblity (#2223) (Siyuan Zhang)
  • [2fbc4ad]: Add local vertex map to arrow fragment and arrow projected fragment. (#2212) (Siyuan Zhang)
  • [cc51704]: Fixes nightly CI on macOS (#2219) (DongZe Li)
  • [f21754f]: [GAE-Java] GRAPE JDK support Spark local mode for graphx (#2216) (Zhang Lei)
  • [ed33b08]: [BugFix] GAE JavaDoc not generated (#2206) (Zhang Lei)
  • [4405721]: Update the DefaultTag to .Chart.AppVersion (#2210) (Ye Cao)
  • [3ee05f7]: [new FAQ] add network proxy related FAQ (#2211) (Yifan Yuan)
  • [09995bf]: Delegate parquet/orc files to vineyard io adaptors (#2209) (Tao He)
  • [7ecd75a]: Delete all deployed resources without a extra rbac role when using helm unintsall (#2208) (Ye Cao)
  • [094e81d]: [Bug Fix] Add default tag (.Chart.AppVersion) to graphscope-store (#2205) (Ye Cao)
  • [4279b8f]: [BugFix] make ResultCode in ir core ffi consistent with Compiler (#2188) (BingqingLyu)
  • [7cfe9a3]: [Bug Fix] Avoid throwing exception after processing grpc error from executor in compiler (#2182) (shirly121)
  • [abdd5c9]: [GAE-Java] Update doc (#2199) (Zhang Lei)
  • [8de000f]: [GAE-Java] Remove String-specialized FFI classes and Simplify Java MessageManager interface (#2195) (Zhang Lei)
  • [5a4312d]: Fix failure in network forward CI (#2204) (Siyuan Zhang)
  • [7735451]: Fixes failures in nightly CI (#2201) (Tao He)
  • [264cbc8]: Log level can accept int and string ignore case (#2200) (Siyuan Zhang)
  • [fb12c7b]: Fix slf4j version conflicts (#2196) (Siyuan Zhang)
  • [87b9602]: [GIE Deploy] Deploy GIE with exp or v6d in standalone mode by helm configuration management (#2143) (shirly121)
  • [ecd58a2]: Fix possible java sandbox protection. (#2191) (Siyuan Zhang)
  • [38a3781]: Enable HDFS tests in CI. (#2175) (Tao He)
  • [7fdcd23]: [GAE-Java] Support grape-graphx running on yarn cluster and support multiple executor on one host (#2185) (Zhang Lei)
  • [f983bd6]: [GIE pegasus] Support cancelling query. (#2168) (Neng Li)
  • [26f0382]: gss ci use lesser image size (#2186) (Siyuan Zhang)
  • [2b00169]: [GAE-Java] Fix GAE-Java driver app's message strategy specification (#2178) (Zhang Lei)
  • [f4dea3e]: [OSPP] Spark RDD Reader for GraphScope (#2103) (Issac-Newton)
  • [cfb707d]: Enhance grpc finding in GAE cmake (#2184) (Tao He)
  • [25ab466]: Use pre-built llvm instead of build when making analytical docker (#2183) (Siyuan Zhang)
  • [79a08d3]: [GIE Compiler] update ResultCode to be consistent with ir core interface (#2158) (shirly121)
  • [74af74f]: [GAE-Java] GraphX-on-GraphScope : remove GraphXFragment and related components (#2123) (Zhang Lei)
  • [49ae814]: [GAE-Java] Add necessary configurations in pom.xml to publish GRAPE jdk to maven central repo (#2166) (Zhang Lei)
  • [2821169]: Fix create interactive engine failure on helm installed graphscope (#2171) (Siyuan Zhang)
  • [89a4a21]: [GAE-Java] GRAPE jdk submodules naming to same pattern (#2163) (Zhang Lei)
  • [321d938]: Upgrade GitHub actions versions (#2160) (Siyuan Zhang)
  • [695cb9c]: Reorganized coordinator and redesigned the logic to create engines. (#2152) (Siyuan Zhang)
  • [ea4ca1d]: fix src label and dst label of edge kinds may be empty (#2153) (Siyuan Zhang)
  • [1b312af]: Move the tolerances of graphscope-store charts to the global scope (#2151) (DongZe Li)
  • [37fa85a]: [GAE-Java] Speed up Java App runtime codegen& compiling (#2150) (Zhang Lei)
  • [733fda3]: [GIE IR] Add docker files to build images of GIE (using exp store) (#2142) (shirly121)
  • [0c5a4b5]: [GAE-Java] Fix : can not compile the submodule of GAE Java SDK (#2146) (Zhang Lei)
  • [3220a7f]: Fixes the conditions for GAE CI in PR and on main, add timeout for test_java_app.py (#2148) (Tao He)
  • [998e361]: Rename pegasus.hosts to network.servers for executor (#2147) (Siyuan Zhang)
  • [97eec70]: Add pytest-timeout to requirements-dev.txt and fixes a deprecated warnings when running tests (#2145) (Tao He)
  • [3038b89]: [GAE-Java] Introduce @GrapeSkip to support generated method overloading (#2144) (Zhang Lei)
  • [77b4ec5]: [GAE-Java] Fix log4j jar missing in grape-runtime and java pie driver app's message strategy settable. (#2140) (Zhang Lei)
  • [7d351a2]: [IR Compiler] make gremlin server listen to 0.0.0.0 instead of localhost (#2141) (shirly121)
  • [759b7a5]: Server now use aks provided by client side (#2132) (Siyuan Zhang)
  • [5c963f5]: Update cluster.py to fix the bug that etcd selector could not work (#2139) (changyi1223)
  • [6d94c3f]: [Bug Fix] Fix bug in index_scan on Groot/Vineyard (#2129) (BingqingLyu)
  • [d4d942c]: [Assembly] wait_servers_ready() instead of sleep 3s when start engine on vineyard (#2136) (BingqingLyu)
  • [2aa3cfc]: Update commons-codec:commons-codec 1.11 to 1.13 (#2119) (zhoumengyks)
  • [6d3d6aa]: Update org.apache.kafka:kafka-clients 2.5.0 to 2.7.2 (#2120) (zhoumengyks)
  • [6d5d288]: Fixes the docstring rendering for GAE apps (#2131) (Tao He)
  • [950ed7a]: App now can carry cmake options, and enabled wcc on string oid. (#2130) (Siyuan Zhang)
  • [31829fd]: [GIE/Runtime] Refine some error infos in Runtime (#2113) (BingqingLyu)
  • [d16b1b4]: Fixes configuration in dockerignore (#2126) (Tao He)
  • [71c37fe]: Upgrade com.fasterxml.jackson.core:jackson-databind to version 2.14.0-rc1 (#2122) (Siyuan Zhang)
  • [ace93c5]: [Pegasus] minor fix: avoid iter.next() when it is None in release (#2112) (BingqingLyu)
  • [5756de3]: Make gae CI invoked when CMakeLists.template changes (#2101) (Zhang Lei)
  • [b6d68bc]: New predicate support (#2086) (waruto)
  • [eb393da]: Bump protobuf-java from 3.18.2 to 3.19.6 in /interactive_engine (#2109) (dependabot[bot])
  • [0b10f5c]: Bump protobuf-java from 3.18.2 to 3.19.6 in /interactive_engine/compiler (#2111) (dependabot[bot])
  • [b1132cc]: [ASoC] report for subtask (#2107) (Haochen Shi)
  • [751e3e7]: [GIE IR] Provide expr store ffi interfaces to encode and write data (#2098) (shirly121)
  • [91db033]: [IR Runtime] Support removing columns in Record via Auxilia in IR-Runtime (#2085) (BingqingLyu)
  • [e024c65]: [GIE-IR] Fix the doc inconsistency issue of the legacy MaxGraph and the latest Gaia-IR. (#2074) (shirly121)
  • [cf7495c]: Pre-release v0.18.0 (#2104) (DongZe Li)
  • [5aabc4d]: [IR Benchmark] Benchmark for BI queries on GIE-IR (#2081) (BingqingLyu)
  • [be1d7a8]: upgrade rocksdb-rust to 0.19.0 (#2100) (Siyuan Zhang)

New Contributors

Full Changelog: v0.17.0...v0.18.0

Contributors

Issac-Newton, changyi1223, and 3 other contributors
Assets 2
Loading