Skip to content

Releases: alibaba/GraphScope

Release v0.28.0

03 Jul 12:48
@lidongze0629 lidongze0629
v0.28.0
a662886
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.28.0 Latest
Latest

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.28.0 release. This release contains many important features and improvements to GraphScope FLEX、gsctl command-line utility、Graph Interactive Engine (GIE) and Groot persistent storage.

We highlight the following improvements included in this release:

1. FLEX Interactive

New functionality:

Refactor

  • Clean the data directory if bulk loading fails
  • Adapt to the new schema and data type definition
  • Fix signal handling and compiler launching
  • Upgrade arrow version
  • Refactor and reorganize the stored procedure interface.

2. Enhancements for GIE

Integrated GOpt, a new graph-native query optimizer, into GIE to boost query performance, featuring:

  • Various heuristic optimization rules in Rule-Based Optimization (RBO), such as NotMatchToAntiJoinRule,FilterIntoJoinRule,FilterMatchRule, ExpandGetVFusionRule etc.
  • A new Cost-Based Optimizer (CBO) leveraging high-order statistics for optimal plan selection.
  • Enhanced s-t pathfinding with CBO to enable more efficient bidirectional searches.

More details can be reffered in the paper and doc.

Support some new functionality, including:

  • elementId() to obtain the inner id of graph vertices or edges in Cypher and Gremlin
  • support CASE WHEN in Cypher
  • values() or valueMap() to obtain properties of each element in path in Gremlin

More details can be referred in Cypher Support Doc and Gremlin Support Doc.

3. Enhancements for Groot

  • Support add properties to existed label #3887

Docker Images

# Flex Interactive image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive:0.28.0

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.28.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.28.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.28.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.28.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.28.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.28.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • fix(python): filetype should be in read_options by @siyuan0322 in #3683
  • fix(interactive): Introduce Calcite-Based CBO Optimizer in GIE Compiler by @shirly121 in #3672
  • fix(interactive): Support Intersect with PathExpand in GIE Runtime by @BingqingLyu in #3600
  • feat(learning): Integrate GLTorch with GraphScope in Server-Client Mode with K8S Deployment Supports by @husimplicity in #3624
  • fix(interactive): Support Gremlin Subquery in Calcite-Based IR Layer by @shirly121 in #3636
  • fix(interactive): Support Optional Match in Graph Optimizer by @shirly121 in #3687
  • Pre-release v0.28.0 by @lidongze0629 in #3686
  • fix(interactive): add MovieTest in the new compilation stack by @BingqingLyu in #3690
  • feat(interactive): Support stopping query service by @zhanglei1949 in #3698
  • refactor(interactive): Return the correct http code by @zhanglei1949 in #3700
  • refactor(interactive): Support Arithmetic Operations on Temporal Types by @shirly121 in #3701
  • feat(interactive): support adding a store pod as a backup by @siyuan0322 in #3703
  • fix(interactive): Concurrent compact by @bufapiqi in #3706
  • fix: Add mocking examples for Portal CI test by @lidongze0629 in #3691
  • fix(interactive): Clean data_dir when bulk_loader fails or cancelled by @zhanglei1949 in #3705
  • Update README.md by @wenyuanyu in #3714
  • chore: fix CI problems by @siyuan0322 in #3712
  • refactor(interactive): Introduce MetaDataStore for Interactive and provide file-based implementation by @zhanglei1949 in #3696
  • fix(interactive): Fix Bug of Duplicated Plan Id Generation After Failover by @shirly121 in #3710
  • fix(interactive): Make rust log location conform to LOG_DIR environment variable. by @siyuan0322 in #3716
  • fix(analytical): make clustering support undirected graph by @acezen in #3715
  • fix(learning): Update graphlearn-torch image version by @husimplicity in #3721
  • build(python): adjust the networkx version by @haoxins in #3463
  • fix(interactive): Integrate multi-pod and one-pod charts into one. by @siyuan0322 in #3720
  • feat(python): Unify the graph level load_from and save to API & Bump up vineyard to v0.22.0 by @acezen in #3610
  • fix(interactive): Support Gremlin Match in Calcite-Based IR Layer by @shirly121 in #3694
  • fix(interactive): Identify pk when it is fused with other predicates in IRCore by @BingqingLyu in #3723
  • fix(interactive): add a simpler PhysicalPlanPrinter in Insight Runtime by @BingqingLyu in #3733
  • fix(interactive): Fix Interactive CI test by @zhanglei1949 in #3738
  • refactor(interactive): Add plugin loading test by @zhanglei1949 in #3743
  • refactor(interactive): add plugin into graph meta by @zhanglei1949 in #3744
  • feat(interactive): Adapt to new data type definition by @zhanglei1949 in #3749
  • feat(interactive): Adapt to new unified schema by @zhanglei1949 in #3725
  • fix(interactive): Make sure admin service always return response in application/json by @zhanglei1949 in #3752
  • chore: adapt to unified schema by @siyuan0322 in #3750
  • refactor(interactive): Remove is_builtin in GraphMeta by @zhanglei1949 in #3754
  • fix(interactive): Fix some implementation of MetaService by @zhanglei1949 in #3759
  • refactor(interactive): Support elementId Operator to Get Node Identifier in Cypher by @shirly121 in #3758
  • fix(learning): Automate graphlearn-torch docker-build by @husimplicity in #3753
  • feat(interactive): Introduce Interactive Java/Python SDK by @zhanglei1949 in #3602
  • fix(interactive): add error handling during file operations by @liulx20 in #3763
  • fix(analytical): Fix MessageStrategy customization for java apps by @zhanglei1949 in #3762
  • fix(interactive): Fix Bugs of count() in Gremlin Based on Calcite-IR Layer by @shirly121 in #3751
  • fix(interactive): Fix waiting compiler ready by @zhanglei1949 in #3770
  • fix(interactive): Fix Bugs of Where Subquery in Gremlin by @shirly121 in #3757
  • fix(interactive): put pegasus hosts to configmap.yaml by @siyuan0322 in #3777
  • fix(interactive): Fix interactive doc and image building by @zhanglei1949 in #3767
  • chore: Export realtime write and submitting query metrics in groot by @siyuan0322 in #3776
  • feat(interactive): Implement API get_graph_schema for AdminService by @zhanglei1949 in htt...
Read more

Contributors

yecol, LiSu, and 15 other contributors
Assets 2
Loading

Release v0.27.0

29 Mar 13:46
@lidongze0629 lidongze0629
v0.27.0
b2db3bd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.27.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.27.0 release. This release contains many important features and improvements to GraphScope FLEX、gsctl command-line utility、Graph Interactive Engine (GIE) and Groot persistent storage.

We highlight the following improvements included in this release:

1. gsctl command-line utility

gsctl is a command-line utility designed to simplify the usage of GraphScope. With gsctl, users can easily interact with GraphScope Flex product, such as Interactive(for high QPS queries) and Insight(for online BI analysis), build binaries and docker images, and perform various utility tasks related to GraphScope. See more detailed informaton from doc.

Key Features of gsctl

  • Utility Scripts: gsctl provides a collection of utility scripts that simplify common tasks such as installing dependencies, building binaries and docker images, and running test suites.
  • Client/Server Mode: With the client/server mode, users can connect to a launched coordinator and interact with specific GraphScope products behind it. This mode enables managing sessions, resources, and other product-specific functionalities.
  • Scope Management: gsctl allows users to switch between global and local scopes, enabling them to work with graphs and resources at different levels. This flexibility enhances productivity and enables efficient management of graph computation tasks.
  • Configuration Management: gsctl includes configuration options that can be customized to suit specific requirements. Users can specify a custom coordinator endpoint and manage other configuration settings as needed.

2. GraphScope FLEX

  • Support build docker images through gsctl flexbuild command.
$ git clone https://github.com/alibaba/GraphScope.git && cd GraphScope
$ python3 gsctl.py flexbuild insight/interactive --app docker
  • Support deploy Interactive and Insight product through docker image.
$ docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex
# launch engine and coordinator service: 12312 gremlin endpoint, 55556 grpc endpoint, 8080 coordinator endpoint
$ docker run -p 12312:12312 -p 55556:55556 -p 8080:8080 registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex /usr/local/groot/bin/start_local_cluster.sh
  • Connect to a launched coordinator and interact with the products
graphscope@host:/work/python$ gsctl connect --coordinator-endpoint http://127.0.0.1:8081
[SUCCESS] Connected to http://127.0.0.1:8081, coordinator is serving with GRAPHSCOPE_INSIGHT mode.

Try 'gsctl --help' for help.

graphscope@host:/work/python$ gsctl ls
schema
├── vertex types
│   ├── person
│   │   ├── Property(name: id, type: LONG, is_primary_key: True)
│   │   ├── Property(name: name, type: STRING, is_primary_key: False)
│   │   └── Property(name: age, type: LONG, is_primary_key: False)
│   └── software
│       ├── Property(name: id, type: LONG, is_primary_key: True)
│       ├── Property(name: name, type: STRING, is_primary_key: False)
│       └── Property(name: lang, type: STRING, is_primary_key: False)
└── edge types
    ├── (person) -[created]-> (software)
    │   ├── Property(name: edge_id, type: LONG, is_primary_key: False)
    │   └── Property(name: weight, type: DOUBLE, is_primary_key: False)
    └── (person) -[knows]-> (person)
        ├── Property(name: edge_id, type: LONG, is_primary_key: False)
        └── Property(name: weight, type: DOUBLE, is_primary_key: False)

3. Enhancements for Groot

  • Integrate a prototype of OpenTelemetry into Groot, users could switch on option trace.enabled=true, which will have a jaeger container that collect traces. And it will serve a website on POD_IP:16686.

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.27.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.27.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.27.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.27.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.27.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • feat(flex): refactor memory management of mmap_array to support Hugepage. by @luoxiaojian in #3505
  • fix(interactive): Fix install_dependency.sh by @zhanglei1949 in #3500
  • fix(flex): Fixed the usage of storage strategy. by @luoxiaojian in #3506
  • fix(interactive): Fix building flex with -DBUILD_HQPS=OFF by @zhanglei1949 in #3497
  • fix(interactive): 离线全量导入时, 新增wait参数, 在download全部完成后, wait指定时间(单位ms), 再进行提交 by @bufapiqi in #3509
  • ci: Fix ci failure and pre-release v0.27.0 by @lidongze0629 in #3512
  • feat(flex): Configure the memory strategy with memory-level. by @luoxiaojian in #3511
  • refactor: coordinator and gsctl under FLEX architecture by @lidongze0629 in #3481
  • chore(deps): bump werkzeug from 0.16.1 to 2.3.8 in /flex/coordinator by @dependabot in #3518
  • chore(deps): bump flask from 2.1.1 to 2.2.5 in /flex/coordinator by @dependabot in #3519
  • docs(interactive): Update doc by @zhanglei1949 in #3514
  • ci: fixes a syntax error in CI by @sighingnow in #3521
  • fix(interactive): Fix argument passing in hqps_app by @zhanglei1949 in #3522
  • chore: change manylinux2014-ci to manylinux2014 by @siyuan0322 in #3529
  • feat(flex): Implemented Immutable Csrs. by @luoxiaojian in #3527
  • fix(interactive): Fix interactive building Dockerfile by @zhanglei1949 in #3528
  • fix(interactive): Fix error when loading edge with string properties by @zhanglei1949 in #3523
  • feat(interactive): Update graph graph_algo by @zhanglei1949 in #3486
  • fix(interactive): support non-blocking data loading interface in coordinator and fix failure during flexbuild process by @lidongze0629 in #3530
  • feat(interactive): Decouple ingestor and store, remote direct RPC connections. by @siyuan0322 in #3525
  • Update README.md for SIGMOD 2024 paper by @wenyuanyu in #3534
  • fix(interactive): Integrate Gremlin Into New Compilation Stack by @shirly121 in #3332
  • fix(k8s): Bump up vineyard to v0.20.3 by @dashanji in #3532
  • fix(interactive): Fix sinking ElementID for TwoLabelVertexSet by @zhanglei1949 in #3535
  • fix(interactive): Fix PathExpand related bugs by @zhanglei1949 in #3538
  • fix(interactive): fix install_dependencies by @liulx20 in #3539
  • fix(flex): Fix warning when compiling with AOCC compilers by @zhanglei1949 in #3510
  • fix(interactive): Fix test_gremlin_timeout in Python Unit Tests by @shirly121 in #3536
  • fix(interactive): Fix Bugs of Group Returning Results from Compiler by @shirly121 in #3533
  • ci: fixes some dependencies issues by @sighingnow in #3553
  • chore: Spelling comments by @jsoref in #3549
  • docs: fixes typos in documentation by @jsoref in #3554
  • fix(analytical): Fix Louvain algorithm's move on const message by @songqing in #3551
  • docs: fixes typos in YAML configurations by @jsoref in #3555
  • feat: Impl the file upload interface for FLEX dataloading by @lidongze0629 in #3557
  • fix(interactive): support src and dst of an edge with different type by @liulx20 in #3552
  • feat(interactive): Support PTHash indexer by @liulx20 in #3558
  • fix(interactive): Fix secondary catchup error by reopening secondary by @siyuan0322 in #3567
  • fix(interactive): Fix a bug in pkscan with expected values given in Within by @BingqingLyu in #3540
  • fix(analytical): fix bug that flash app can not run on string oid graph by @acezen in #3570
  • fix: Spelling python graphscope tests by @jsoref in https://gi...
Read more

Contributors

LiSu, jsoref, and 15 other contributors
Assets 2
Loading

Release v0.26.0

25 Jan 03:10
@lidongze0629 lidongze0629
v0.26.0
6db110e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.26.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.26.0 release. This release contains many important features and improvements to GraphScope FLEX and Graph Interactive Engine (GIE), including secondary instance and type inference in GIE, loading from distributed data source and Macs deployment in FLEX Interactive.

We highlight the following improvements included in this release:

1. FLEX Interactive

  • Support use GraphScope Interactive on macos. Download the latest release to give it a shot interactive-latest
  • Support bulk loading with odps table as data source. Here is an example
graph: dd_graph
loading_config:
  data_source:
    scheme: odps  # file, odps
  import_option: init # init, overwrite
  format:
    type: arrow
vertex_mappings:
  - type_name: Person  # must align with the schema
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
  column_mappings:
      - column:
          index: 0  # can be omitted if the index is the same as the property index
          name: user_id  # can be omitted if the name is not known
        property: id
edge_mappings:
  - type_triplet:
      edge: Knows
      source_vertex: Person
      destination_vertex: Person
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
    source_vertex_mappings:
      - column:
          index: 0
          name: src_user_id
        property: id
    destination_vertex_mappings:
      - column:
          index: 1
          name: dst_user_id
        property: id

2. Enhancements for GIE

  • Support secondary instance in Groot. It enables user open another read-only instance along with the original instance, providing additional read qps to users.

    • How to use: Set secondary.enabled=true in helm charts to enable secondary mode. Other options could remain unchanges with the primary instance.
    • Note in secondary instance it could only be queried upon, the modification of schema and data should be routed to primary instance.
    • See more details in the doc

  • Support automatic type inference in graph queries, which can affect query execution in two ways:

    • Checking if vertex and edge types in the graph conform to user-defined schemas 
      # Now compiler can infer that 'kno' is a nonexistent edge type and returns error
Match (a:person)-[b:kno]->(c) Return a, b, c;
    • Inferring vertex and edge types in the graph. 
      # Now compiler can infer a and b must both be person types, and execution can be more efficient
Match (a)-[:knows]->(b) Return labels(a), labels(b)
=> Match (a:person)-[:knows]->(b:person) Return labels(a), labels(b)

See more details in the doc

3. Other enhancements and bug fixes

  • Introduce graph_algo as another example graph. https://github.com/alibaba/GraphScope/tree/main/flex/interactive/examples/graph_algo.
  • Support String DT_STRING as edge property type.
  • Refactor Flex/CMakeLists.txt to support use installed flex via find_package(flex).
  • Fix decode error in Groot
  • Fix memory leak in Pegasus and data race in Groot
  • Fix GIE peers_contains bugs in count operator in subtasks
  • Fix Pegasus unexpect result of operator aggregate() + iterate()
  • Refine Log Printing Messages in Compiler
  • Fix bugs of Groot bulk loading when it retry to download source files
  • Fix bugs in GIE for edge filtering

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.26.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.26.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.26.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.26.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.26.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.26.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • ci: fixes failure during pushing image and pre-release v0.26.0 by @lidongze0629 in #3339
  • fix(interactive): Fix decode error in groot by @siyuan0322 in #3340
  • fix(interactive): Check vertex existence whenever queried by @siyuan0322 in #3343
  • feat(interactive): Support more data types for vertex/edge property by @zhanglei1949 in #3346
  • feat(interactive): Support fold_partiton_by_key by @lnfjpt in #3324
  • fix(interactive): fix a memory leak bug in IR Core by @BingqingLyu in #3349
  • fix(interactive): replace ok_or() by ok_or_else() for lazy evaluation by @BingqingLyu in #3350
  • feat(flex): support LFIndexer resize by @liulx20 in #3351
  • refactor(interactive): Refactor CSVFragmentLoader to remove redundant code by @zhanglei1949 in #3352
  • fix(interactive): Fix support for PathExpand for multiple triplets. by @zhanglei1949 in #3335
  • fix(interactive): Remove Unnecessary JNA Structure Creation to Save Memory Consumption by @shirly121 in #3354
  • fix(interactive): extension use hash(srcId, dstId, edgeLabelId, edgePks) instead of increase eid by @bufapiqi in #3357
  • fix(interactive): Comparison between std::shared_ptr<arrow::DataType> should use Equals() by @zhanglei1949 in #3361
  • chore: fixes the missing tzdata issue in graphscope ubuntu-22.04 based images by @sighingnow in #3363
  • chore: make sure tzdata can be installed by @sighingnow in #3365
  • fix(interactive): fix memory leak in pegasus by @lnfjpt in #3367
  • fix(interactive): add some optimizer microbenchmark queries by @BingqingLyu in #3369
  • fix(interactive): Fix memory leak in pegasus and data race in groot by @siyuan0322 in #3371
  • chore: Add writer interface for groot by @siyuan0322 in #3373
  • fix(interactive): Add a new release-with-debug-info profile by @siyuan0322 in #3372
  • fix(interactive): fix the bug in filtering after intersection in GIE Runtime in distributed computation by @BingqingLyu in #3359
  • feat(interactive): Add Rate Limiter for Control Flow by @shirly121 in #3368
  • refactor(interactive): Improve the code of checking if a given label sets contain all labels in the schema by @longbinlai in #3374
  • feat(interactive): Implement admin http service for hqps engine by @zhanglei1949 in #3322
  • fix(interactive): Support Start Tag for Graph Operators in Compiler by @shirly121 in #3376
  • fix(interactive): Fix documentation for interactive. by @zhanglei1949 in #3384
  • docs: Fix GIE Benchmark Readme bug by @JackyYangPassion in #3389
  • feat(interactive): Enable replay WAL records from offset of timestamp in groot by @siyuan0322 in #3385
  • refactor(interactive): flex storage with mmap array by @liulx20 in #3356
  • fix(interactive): event message error by @lnfjpt in #3380
  • fix(interactive): fix peers_contains bugs in count operator in subtasks by @lnfjpt in #3390
  • fix(interactive): fix unexpect result of operator aggregate() + iterate() by @lnfjpt in #3391
  • fix(interactive): Fix Start Time Metric in Compiler Log by @shirly121 in #3393
  • fix(interactive): Refine Log Printing Messages in Compiler by @shirly121 in #3381
  • feat(interactive): Introduce explicit barrier for actors when switching query service to a different graph by @zhanglei1949 in #3395
  • feat(interactive): Add support for loading graph with odps table as data source by @zhanglei1949 in #3305
  • refactor(interactive): Support Logical Operator within in Index Predicate by @shirly121 in #3398
  • feat(interactive): support string as edge property type by @liulx20 in #3403
  • fix(interactive): Pass acid by @liulx20 in #3415
  • fix(python): fixes the result validation for edge sets by @sighingnow in https://github.com/alibaba/GraphScope/pull...
Read more

Contributors

longbinlai, sighingnow, and 12 other contributors
Assets 4
Loading

Release v0.25.0

10 Nov 07:13
@lidongze0629 lidongze0629
v0.25.0
8235b29
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.25.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.25.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), FLEX Interactive, and bug fix.

1. Interactive under GraphScope Flex Architecture

  • Release GraphScope Interactive.
  • Enrich the movie graph schema and example query statements.
  • Implement a query cache to avoid recompiling the same adhoc query.
  • Support using the property of string type as primary key for vertex.

2. Enhancements for GIE

  • Support get disk status in groot， user could retrieve the total space and free space of the store pod via Python/Java client. Example API is here
  • Enhancements to count(): We've implemented two primary optimizations to accelerate the count() operation:
    • Optimized Counting of Vertices/Edges: Operations like g.V().count() that previously required a complete enumeration of vertices followed by a count, are now streamlined into a single fused operator. This operator efficiently counts vertices or edges in tandem with their iteration.
    • Improved Worker Utilization: The prior count() implementation was limited to using a single worker per machine for counting tasks, irrespective of the available configured workers. Now, count() operations are distributed across all configured workers, leveraging parallel processing to boost performance.
  • Users can now interact with GIE using natural language. We've integrated the capability to call OpenAI-compatible APIs within GIE, enabling the translation of human language inputs into Cypher queries. This advancement allows for a more intuitive and accessible query experience. For instance, users can utilize our Python API to execute the following code:
from graphscope.langchain_prompt.query import query_to_cypher
endpoint = "https://xxx" # use your endpoint 
api_key = "xxx" # replace to your own api key
question = "贾宝玉是谁的儿子?"
cypher_sentence = query_to_cypher(graph, question, endpoint=endpoint, api_key=api_key)
print(cypher_sentence)

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.25.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.25.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.25.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.25.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.25.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.25.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • refactor(interactive): Generate Stored Procedure Config Automatically in Compiler by @shirly121 in #3140
  • feat(interactive): Support length() Operator in Cypher Queries by @shirly121 in #3166
  • fix(interactive): Fix bug in computing path length in GIE by @BingqingLyu in #3172
  • ci: Support to build GraphScope Images(both x86 and arm64) in CI workflow by @FLYLX in #3152
  • ci: fixes failure of building images by adding actions/checkout step by @lidongze0629 in #3174
  • ci: Pre-release v0.25.0 by @lidongze0629 in #3175
  • chore: fixes the link for the flex directory by @sighingnow in #3176
  • fix(interactive): Fix the bug in the RexNode to Proto Conversion by @shirly121 in #3171
  • feat(interactive): Support [] Operator in Cypher Queries by @shirly121 in #3168
  • refactor(interactive): Parse Graph Schema from Yaml Configuration by @shirly121 in #3145
  • feat(python): Refactor coordinator for operator by @siyuan0322 in #3148
  • fix(interactive): don't require openssl-sys during building GIE by @sighingnow in #3181
  • feat(interactive): Enable clear properties for groot node by @siyuan0322 in #3180
  • refactor(interactive): Support Nested Branch Logical Plan by @MeloYang05 in #3151
  • refactor(interactive): Support head(collect(XX)) in Cypher Queries by @shirly121 in #3178
  • fix(python): fix default field in py311 by @siyuan0322 in #3183
  • feat(interactive): Support IS_NULL and IS_NOT_NULL in Cypher Queries by @shirly121 in #3167
  • feat(interactive): Support Unfold step in GIE by @Louyk14 in #3143
  • remove google-java-format.jar by @Louyk14 in #3196
  • feat(interactive): Add schema management ability for groot java sdk by @siyuan0322 in #3193
  • refactor(interactive): Support DateFormats in GIE Runtime by @BingqingLyu in #3197
  • fix(interactive): Fix bugs of Chinese Encoding by @shirly121 in #3195
  • refactor(flex): Replace the Adhoc csv reader with Arrow CSV reader by @zhanglei1949 in #3154
  • refactor(interactive): Support Date Type and EXTRACT Operator in Cypher Queries by @shirly121 in #3169
  • ci: Update pr-check.yml by @yecol in #3207
  • feat(interactive): support project properties of a path by @BingqingLyu in #3213
  • feat(interactive): Enhancement pieces for groot by @siyuan0322 in #3217
  • fix(interactive): Fix bugs in Label Name to Id Conversion by @shirly121 in #3221
  • feat(interactive): Support LDBC SNB IC1,4,7,10 for HQPS Engine by @zhanglei1949 in #3147
  • chore: don't reinstall (may trigger upgrade) if libarrow-dev has already been installed by @sighingnow in #3241
  • ci: fixes the Python dependency error in CI by @sighingnow in #3242
  • fix(interactive): Not requiring eid in update/delete edges in groot by @siyuan0322 in #3240
  • fix(python): default session needs to be aware of g.set_option(num_workers=...) by @sighingnow in #3246
  • refactor(interactive): Refactor the user interface of Interactive by @zhanglei1949 in #3201
  • feat(interactive): support unfold path entry in GIE Runtime by @BingqingLyu in #3236
  • fix(interactive): Fix several bugs in data-load-tools by @siyuan0322 in #3249
  • fix(interactive): Fix NPE when update/delete vertices by @siyuan0322 in #3250
  • build(interactive): Reduce image size by @zhanglei1949 in #3247
  • fix(interactive): Support fusion of scan+count in GIE Runtime for optimization by @BingqingLyu in #3233
  • fix(flex): Support set edge strategy on each vertex_type_pair_relation by @zhanglei1949 in #3257
  • feat(learning): Integrate GLTorch into GraphScope by @LiSu in #3230
  • feat(interactive): Support labels and type Functions in Cypher Queries by @shirly121 in #3255
  • build(learning): bump torch from 1.13 to 1.13.1 in /python by @dependabot in #3264
  • refactor(interactive): Introduce a new test set for Interactive by @zhanglei1949 in #3251
  • docs(interactive): Add docs for GraphScope Interactive by @zhanglei1949 in #3267
  • fix: Flex add rt_bench and fix some bug by @liulx20 in #3268
  • chore: Enable retrieve disk status of store node from client by @siyuan0322 in #3274
  • fix(interactive): clear download path when start store service by @siyuan0322 in #3276
  • ci: requires macos 12 as homebrew (and apple) do not provide support for this old version by @sighingnow in #3278
  • fix(interactive): Fix Implicit Type Conversion Issues in HQPS by @shirly121 in #3256
  • fix(interactive): block service until graph schema is synced by @siyuan0322 in #3280
  • chore: Increase the disk size of minikube cluster by @siyuan0322 in #3283
  • fix(intera...
Read more

Contributors

yecol, LiSu, and 15 other contributors
Assets 2
Loading

Release v0.24.0

01 Sep 10:14
@lidongze0629 lidongze0629
v0.24.0
8950371
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.24.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.24.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), Learning Engine(GLE), and Deployment.

We highlight the following improvements included in this release:

1. Enhancements for GIE

Key Features:

  • Support the sample() step of Gremlin to randomly pick up a given number of traversers from a Gremlin traversal. Additionally, coin() step can now follow any step in a Gremlin traversal, instead of just V() and E(). For example,
g.V().sample(10)   // Sample 10 traversers from `g.V()`
g.V().out().coin(0.2)  // Randomly pick up 20% among all vertices' out neighbors
  • Support the union step of Gremlin to combine the traversers from multiple traversals. For example,
g.V().union(out(), out().out())  // the results would now be one-hop (out once) and two-hop (out twice) neighbors of all vertices.
  • Support the unfold step of Gremlin, which is a reversed operator of fold that flattens a collection of traversers into discrete elements. For example,
g.V().fold().unfold()  // must obtain `g.V()` again
  • Support the identity() step of Gremlin, which simply returns the current traversers, and is often used together with union step. For example,
g.V().union(identity(), out())  // return `V()` and its one-hop neighbors
  • Support isNull in the expr() for verifying wether a certain property has NULL value. For example,
g.V().where(expr("@.age isNull"))  // the vertices without the field of `age` will be pruned

But fixes:

  • Fix a bug of count() step followed by order() and limit() that can cause the gaia engine to abort unexpectedly. Now the following Gremlin query can be executed smoothly:
g.V().order().by('id', asc).limit(1).out().count()

2. Enhancements for Deployment

  • All GraphScope Images support both x86 and arm64 platform
  • This release includes the gsctl command-line utility for building and testing GraphScope. The gsctl tool provides several commands and options to streamline the development and deployment process. You can use the following command to install the graphscope-client package and then use gsctl:
$ pip3 install graphscope-client
$ gsctl --help

3. Other enhancements and bug fixes

  • Reduced the size of GRAPE-java shaded jar,reduce shaded jars' total size from 94MB to 58MB.
  • Introduce Hiactor-based High-QPS Engine for Flex.
  • Refine the schema definition for flex rt_mutable_graph
  • Fix the Py_None reference count issue in GLE
  • Fix the bug that GLE client only connects with a single server in distributed training
  • Update the GLE-related tutorials

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.24.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.24.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.24.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.24.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.24.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.24.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

Read more

Contributors

yecol, LiSu, and 20 other contributors
Assets 2
Loading

Release v0.23.0

04 Jul 12:26
@lidongze0629 lidongze0629
v0.23.0
7dcf195
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.23.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.23.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), GraphScope Flex, and Deployment.

We highlight the following improvements included in this release:

1. GraphScope Flex Technical Preview

GraphScope Flex represents the ongoing evolution of GraphScope. In this release, we're excited to introduce a technical preview of GraphScope Flex. It highlights a modular design that reduces resource and cost requirements while providing a seamless, user-friendly experience for flexible deployment. It's currently under active development, and we look forward to your feedback.

Key Features:

  • Modular Design: Assemble your stack much like LEGO blocks to customize your graph computing deployments.
  • Three-Layer Architecture: Components are organized into an application layer, execution layer, and storage layer.
  • Flexible Builds and Deployments: Use the flexbuild script to build a custom deployment tailored for your specific use case.

You can explore GraphScope Flex in the flex/ directory or through the released artifacts

  • graphscope_flex_db_cppsp_hiactor_mcsr,
  • graphscope_flex_olap_builtin_grape-cpu, and
  • graphscope_flex_gnn_gnnmodels_graphlearn_tensorflow_vineyard.so

packages for high-QPS interactive queries, graph analytics, and graph learning task use cases, respectively. Dive in and discover what GraphScope Flex has to offer!

2. Enhancements for GIE

  • Support the recording of both vertices and edges during path expansion. In the past, path expansion had the option to yield either all vertices or the end vertex of the path. However, due to the requirements of both Gremlin and our users, it's essential to also retrieve the edges of the path, besides the vertices. To activate this feature, use with('RESULT_OPT', 'ALL_V_E') in the path expansion syntactic sugar. Here is an example: 
    gremlin> g.V().out("1..3", "knows").with('RESULT_OPT', 'ALL_V_E')
     ==>[v[1], e[0][1-knows->2], v[2]]
     ==>[v[1], e[2][1-knows->4], v[4]]
  • We are now happy to introduce the capability of querying Cypher in GIE, by integrating Neo4j's bolt service in our system. Please follow the guide to enable bolt service (only on local) for querying with Cypher. We also attempt to make the syntax of Cypher as close as openCypher, and the details of our support for Cypher can be found here.

3. Other enhancements and bug fixes

  • GAE Java
    • Fix VertexSet's problem of supporting vertex_id in java long.
    • Add Grape-GraphX performance report
    • Fix the problem of installing grape-jdk locally.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.23.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.23.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.23.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.23.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.23.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.23.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • [7dcf195]: Change the version to v0.23.0 (#2969) (Dongze Li)
  • [7316297]: Introduce flexbuild and a readme to flex. (#2954) (luoxiaojian)
  • [d8ad6ce]: Install jdk 11 on MacOS CI and fixes the GAE link failure in nightly CI on MacOS (#2965) (Tao He)
  • [c27f459]: Fixes two compliation warnings (#2968) (Tao He)
  • [bb1921e]: Update the elementMap-step's doc of GIE (#2966) (Dongze Li)
  • [cb27df4]: Fixes nightly CI error for Arm Linux wheels (#2963) (Tao He)
  • [69258d9]: Fixes the "illegal thread local variable reference to regular symbol" error on macOS for GLE (#2962) (Tao He)
  • [1e2c8f6]: [GIE Compiler] Unify Gremlin Timeout Configurations (#2953) (Xiaoli Zhou)
  • [4694963]: Add setuptools to client requirement in order to use pyproject based build (#2958) (Siyuan Zhang)
  • [7fc14b5]: [GIE Compiler] Support Case When Expression in Logical and Physical Plan (#2918) (Xiaoli Zhou)
  • [b82b748]: Avoid implicit int to enum conversion, causing the -fpermissive error (#2957) (Siyuan Zhang)
  • [ad4aea1]: Use the correct delete in free_graph_handle (#2955) (Tao He)
  • [21fd644]: Sort the server list before iterate to keep consistent between workers (#2956) (Tao He)
  • [502e410]: Parallelize the PrepareToRun implementation in projected fragment (#2949) (Tao He)
  • [5ffad66]: Add two more pagerank variants (#2950) (Siyuan Zhang)
  • [a222479]: Use pagerank_local_parallel rather than pagerank_local for better performance (#2945) (Tao He)
  • [92da638]: Refine CI process (#2948) (Siyuan Zhang)
  • [555b13d]: add figures for GraphScope Flex. (#2946) (Jingbo Xu)
  • [7035c0a]: ElementMap step support (#2942) (Dongze Li)
  • [b6402e7]: cluster_builder.py: add missing copyright header and remove unused import lines (#2944) (Tao He)
  • [5b852ff]: Docs preview is only available for pull requests from the orignal repo, not forks (#2943) (Tao He)
  • [cab8cde]: Add GLE quick start example in the overview getting started page (#2939) (LiSu)
  • [badf3f3]: hotfix: pull_request_target issue. (#2941) (Jingbo Xu)
  • [08f8d0f]: Disable JAVA SDK (with a warning) when JNI not found (#2937) (Tao He)
  • [f543cd2]: Improve the deploy_with_existing_vineyard_cluster doc for easy understanding (#2930) (Ye Cao)
  • [ae8d96c]: Show explicit error message for SSSP run over a graph with no edge weight (#2929) (Lei Wang)
  • [50c7a1e]: Pin vineyard python version to 0.15.0 (#2933) (Siyuan Zhang)
  • [25496f8]: Connect to frontend directly, bypass coordinator for GIE query (#2923) (Siyuan Zhang)
  • [96d5093]: [GIE Compiler] Parse Label Id to Name in label Operator (#2928) (Xiaoli Zhou)
  • [34b3650]: [GAE-Java] Use FFI-bitset rather than java BitSet for VertexSet backend (#2926) (Zhang Lei)
  • [f0a779e]: [good-first-issue]Move save_to/load_from from client side to coordinator (#2917) (Zhang Lei)
  • [c1007cf]: [GIE Compiler] Introduce cypher service to accept queries from neo4j ecosystem (#2848) (Xiaoli Zhou)
  • [7637aaf]: [GIE] Refine FFI interface build_physical_plan (#2911) (Xiaoli Zhou)
  • [d0dc2b7]: Print metrics of application running time (#2908) (Siyuan Zhang)
  • [628773d]: [Bug Fix] Fix bug of null pointer exception in distinct operator (#2901) (Xiaoli Zhou)
  • [0a44e71]: Fixes the misuse of "continue-on-error" and "fail_ci_if_error". (#2904) (Tao He)
  • [612bd7f]: Upload coverage: continue on error to avoid false negative on main branch (#2903) (Tao He)
  • [513155c]: Upgrade the graphlearn submodule to fixes nightly CI, refactor the Makefile for early error exit (#2902) (Tao He)
  • [bd7680f]: Use CONFIG mode to find protobuf to address the absl dependency issue. (#2899) (Tao He)
  • [cf81c7b]: [GAE-Java] Relax version enforce for some mvn dependencies (#2894) (Zhang Lei)
  • [b37ab90]: [GIE Compiler] Add data type of algebra operator output to physical plan (#2870) (Xiaoli Zhou)
  • [79cd577]: Use local metadata backend to avoid the requirements of etcd for local sessions (#2889) (Tao He)
  • [7ef0e4d]: fix munew line are not rendered correclty when multiple params are given (#2891) (Siyuan Zhang)
  • [fbf40f6]: Update the pre-genereated unload_op when the op itself been deepcopied (#2888) (Tao He)
  • [32089f2]: [Grape-Java] Add grape-graphx performance.md (#2886) (Zhang Lei)
  • [e4f1f23]: Allow pass params to gie instance (#2885) (Siyuan Zhang)
  • [17819c8]: [BugFix] Fix bug in endV() after PathExpand (#2881) (BingqingLyu)
  • [a18836d]: [GIE Compiler] Support dynamic query params in cypher queries (#2874) (Xiaoli Zhou)
  • [ce3e16b]: [GAE/tests] Add GAE tests for FLASH algorithms of bfs and cc (#2869) (Longbin Lai)
  • [5a63faf]: upload log should not fail the CI (#2879) (Siyuan Zhang)
  • [559b084]: Direct the content of log files to stdout of pods in kubernetes deployment (#2871) (Siyuan Zhang)
  • [4af5689]: add initial version of command-line tool gsctl. (#2868) (Jingbo Xu)
  • [7a3e06b]: [GIE Doc] Update Doc for PathExpand Step Usage in GIE (#2877) (BingqingLyu)
  • [387f915]: Fixes the docs generation for tagged releases (#2878) (Tao He)
  • [c489c63]: [BugFix] Make the error message truncated not so aggressive (#2876) (Weibin Zeng)
  • [e9af91a]: [GIE] Support PathExpand with OPT=ALL_V_E in GIE (#2841) (BingqingLyu)
  • [3773889]: Add a initializeCommand to devcontainer.json: always pull newer image before create container (#2873) (Weibin Zeng)
  • [086fa63]: [GIE/docs] reorganize gremlin docs (#2862) (Longbin Lai)
  • [bd89520]: Allow triggering workflow dispatch event for CIs (#2865) (Ke Meng)
  • [a0b36ee]: [GIE Doc] Add examples to collect gremlin results in a streaming way (#2842) (Xiaoli Zhou)
  • [5fd6cc1]: Fixes the inconsistent usage of msgpack/json in graph reporter and avoid protobuf message creation in del (#2843) (Tao He)
  • [434acdf]: Fix dns search error in groot (#2850) (Siyuan Zhang)
  • [9bf071c]: auto-gen docs of flex (#2851) (ds-ssj)
  • [6851cd9]: allow null value in data loading (#2845) (Siyuan Zhang)
  • [1ba6102]: [GIE Compiler] Introduce GraphPlanner to unify logical and physical plan b...
Read more
Assets 3
Loading

Release v0.22.0

06 Jun 10:08
@lidongze0629 lidongze0629
v0.22.0
99c9d15
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.22.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.22.0 release. This release contains many important features and enhancements to Interactive Engine (GIE) and Deployment. Now the user can try GraphScope under the Linux Arm64 platform by pip3 install graphscope.

We highlight the following improvements included in this release:

1. Enhancements for GIE

  • We have provided a utility for users to easily develop and test GIE on a local machine. Basically, users can run three parts of integration tests using the gs command line tool, including the official gremlin test, auxiliary pattern match test, and ldbc complex interactive workload on ldbc social network scale factor 1. In addition, users are allowed to locally start a frontend and an executor, where the former exposes an endpoint to easily integrate with the official gremlin console, and the latter is responsible for executing queries. Details can be found in the documentation.

2. Enhancements for Deployment

  • User can try GraphScope under the Linux Arm64 platform by pip3 install graphscope
  • All images are now switched to ubuntu-based.
  • Add a devcontainer.json for users to quickly get started.
  • Use vineyardctl to install vineyard as the sidecar container.
  • Support to deploy the engines of GraphScope on demand in the lazy mode.

3. Other enhancements and bug fixes

  • Support to store and restore graphs from/to persistent storage on the kubernetes cluster. Details can be found in the guide: https://graphscope.io/docs/latest/deployment/persistent_storage_of_graphs_on_k8s.
  • Add an end-to-end example about training over vineyard graphs using GLE
  • Fixes the incorrect problem of results in python algorithm with PIE model under the session's parameter num_workers=1
  • We have fixed a bug of GIE related to using match step with a not-existed edge, for which it should be executed via anti join, but it returned unexpected error "the first sentence of MergedSentence must have InnerJoin" instead.
g.V().match(
    __.as("a").in().as("b"),
    __.as("b").out().as("c"),
    __.not(__.as("c").out().as("a")) # `not`-exsited edge
)

Now such kind of query can be executed without issue.

  • Fixes the bug of incorrect results when dedup edges on a partitioned graph in GIE, e.g.,
g.V().outE().dedup()

Now it can output expected results.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.22.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.22.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.22.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.22.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.22.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.22.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • [99c9d15]: Fixes the GAE fork issue with openmpi (#2820) (Tao He)
  • [ba35996]: [GLE Doc] complete the build and install details of learning engine (#2815) (LiSu)
  • [7af9255]: Fixes some failures in nightly CI (#2817) (Tao He)
  • [0b22053]: Support to install deps for Alibaba Cloud Linux in the gs bash utility (#2816) (Ye Cao)
  • [28a3605]: Cleanup space in CI (#2814) (Siyuan Zhang)
  • [3f50d7d]: Update dev_guide.md: change the link to learning engine to a correct one. (#2813) (Taoshu)
  • [035036b]: disable debug-assert in groot (#2811) (Siyuan Zhang)
  • [57b22fe]: Groot schema now supports add comment (#2802) (Siyuan Zhang)
  • [43493ce]: Fixes bug when building docs from pull requests (#2810) (Tao He)
  • [302b028]: Support to backup and recover the specific graphs via vineyardctl (#2771) (Ye Cao)
  • [d49d0ed]: docs: update rst-versions style, defaults default version to latest, and fixes the zh/en indexing (#2803) (Tao He)
  • [9efd4f6]: Add local python script path to PATH (#2804) (Tao He)
  • [9d5d968]: [Docs/GIE] add tinkerpop ecosystem (#2784) (Longbin Lai)
  • [278a24c]: Adapt to latest vineyard for varint encoding in CSR (#2791) (Tao He)
  • [f1b216c]: Support users to only input the kubeconfig path for connecting the k8s cluster (#2799) (Ye Cao)
  • [33e3007]: Correct the version of built wheel (#2798) (Siyuan Zhang)
  • [191949b]: Fixes the graphdef value when constructing graphs from vineyard_id. (#2793) (Tao He)
  • [699ffbe]: [k8s]Support deploying vineyard cluster independently and deploy the engines on called (#2710) (Ye Cao)
  • [c1ef885]: refine docs latest tag to make it ready as default. (#2788) (Jingbo Xu)
  • [e9090db]: Refine doc for dev and test of analytical engine. (#2787) (Jingbo Xu)
  • [1e57650]: Read version number from the package in setup.cfg (#2785) (Dongze Li)
  • [09ead30]: [GLE docs] minor fixes (#2783) (LiSu)
  • [b014dad]: use kubectl cp --retries (#2781) (Siyuan Zhang)
  • [d2c7d2e]: Fix typo in Makefile (#2782) (Siyuan Zhang)
  • [3606b85]: Fixes the building failure of client wheel on Linux arm64 platform (#2780) (Dongze Li)
  • [fdc626e]: Use vineyardctl API to inject the vineyard sidecar (#2612) (Ye Cao)
  • [7fd430b]: [GraphProxy] refine implementation of LazyDetails (#2766) (BingqingLyu)
  • [4b24146]: [Bug Fix] fix the bug in implementation of PartialEq for Edge in GIE Runtime (#2740) (BingqingLyu)
  • [5b46a68]: Improve GetInnerVertex() by querying local fragment only (#2772) (Songqing Zhang)
  • [983a41e]: Enable deploy groot on local environment (#2769) (Siyuan Zhang)
  • [2509958]: Make grape_engine help message more friendly (#2774) (Siyuan Zhang)
  • [e294377]: [GIE Dev Test] Refine GIE Dev and Test (#2767) (Xiaoli Zhou)
  • [bf8d7b0]: Generate pyi stubs for protobuf modules for better developer experience (#2768) (Tao He)
  • [6fc7db3]: [Bug Fix] Fix the alias of GetV of ExpandBase in PathExpand, an… (#2763) (BingqingLyu)
  • [a119c23]: Correct the python sssp result by setting generate_eid to false in load_p2p_network dataset (#2762) (Dongze Li)
  • [e4bb069]: Make sure grape-engine exit when CTRL-C without crash (#2761) (Tao He)
  • [df16284]: Store address offset of string properties with 48-bit unsigned intege… (#2760) (luoxiaojian)
  • [747df41]: Drop the environment switch in dev and test, and update gs (#2754) (Siyuan Zhang)
  • [46d7ed8]: Fixes duplicated run and upload actions for k8s-ci (#2751) (Tao He)
  • [3cf0798]: Propogate the k8s exception to avoid waiting forever (#2747) (Tao He)
  • [e6af69b]: Ensure use string as log-level, even for logging.XXX (#2745) (Tao He)
  • [e969d0e]: [GIE Dev Test] Add docs for interactive dev test (#2733) (Xiaoli Zhou)
  • [ac7308e]: [BugFix] add Metadata for PathExpand (#2720) (BingqingLyu)
  • [94ec2d8]: Updated license and notice. (#2725) (Jingbo Xu)
  • [8ab7b1b]: Overhaul data-load-tools (#2715) (Siyuan Zhang)
  • [b03bc60]: Publish wheel packages on Linux arm64 platform nightly (#2732) (Dongze Li)
  • [1e63e87]: [GIE/bug] fix a bug of matching with anti join (#2730) (Longbin Lai)
  • [1a81620]: use javac to find the version of java (#2728) (Siyuan Zhang)
  • [51f92ac]: Support to build wheel package on Linux aarch64 platform (#2723) (Dongze Li)
  • [1d3cd4a]: refine retry logic to exponential backoff (#2729) (Siyuan Zhang)
  • [06f8769]: Fixes the installation prefix for nightly linux CI (#2727) (Tao He)
  • [662014f]: Opt-out zstd in features when requiring rocksdb (#2722) (Tao He)
  • [0a5c144]: set timeout of rt_admin to 1 minute. (#2719) (luoxiaojian)
  • [fb79c62]: Bump up graph-learn submodule version (#2716) (Tao He)
  • [7a5ed13]: add docs for using dev container and fix typo (#2711) (Siyuan Zhang)
  • [5bd90ba]: Requires correct version of graphscope_client in coordinator to avoid diverage (#2708) (Tao He)
  • [10a6488]: Introduce the new version of kubernetes log exporter for dumping logs during a period of time such as pytest (#2706) (Ye Cao)
  • [e447cd5]: Fixes the linux wheel install prefix (#2707) (Tao He)
  • [f476c57]: Fixes the 'sudo' in dockerfile for gss image (#2704) (Tao He)
  • [3b11b77]: Build wheels for macOS m1 (#2701) (Tao He)
  • [446b660]: [LICENSE] add dependency licenses for calcite and antlr (#2699) (Xiaoli Zhou)
  • [52b1ee2]: Fixes the initialization of paramters in load_from() (#2698) (Tao He)
  • [d4b72fd]: Display stderr logs as expected on K8s (#2696) (Tao He)
  • [6da5c31]: Add version info for making interactive (#2685) (Siyuan Zhang)
  • [6dd3af2]: Implemented 3-retries, added detailed logs for insert queries, and unified bulk-load and recovery. (#2691) (luoxiaojian)
  • [03165eb]: [FEAT][GraphAr] Add API to python client to enable loading graph from gar files or archive graph to gar files (#2588) (Weibin Zeng)
  • [c2cb286]: Export the k8s logs of graphscope helm test and upload to artifact when the helm test failed (#2683) (Ye Cao)
  • [81230c5]: Add prepare-commit-msg (#2688) (Siyuan Zhang)
  • [90b7e00]: Update install-hook.sh (Wenyuan Yu)
  • [ff9582e]: Create install-hook.sh (#2687) (Wenyuan Yu)
  • [909b3c4]: [GIE/Runtime] Remove the dependency of GIE Runtime on GlobalQueryStore to accelerate the compilation (#2681) (BingqingLyu)
  • [a46663e]: Clearify the requirements for MacOS (#2684) (Tao He)
  • [25a70f1]: Preload searchindex.js in background to optimize the search experiences (#2682) (Tao He)
  • [b258999]: Bump up vineyard to v0.14.5 (#2676) (Siyu...
Read more

Contributors

TaoLbr1993 and wenyuanyu
Assets 2
Loading

Release v0.21.0

25 Apr 13:55
@lidongze0629 lidongze0629
v0.21.0
2657bec
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.21.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.21.0 release. This release contains many important features and improvements to GraphScope. The Graph Learning Engine (GLE) now supports the 'SubGraphSampler' and 'RandomWalk' operators and adds a node labeling-based SEAL and node2vec model. Also, GLE supports node cache for accelerating distributed feature lookup. This release also adds a 'devcontainer.json' for users to easily set up a developing environment using VSCode.

We highlight the following improvements included in this release:

  • Enhancements for GLE:
    a. Add operators: SubGraphSampler, RandomWalk
    b. Add node labeling based SEAL model, node2vec model.
    c. Support temporal graph with the timestamp, and add TokTimestampSampler operator.
    d. Add temporal models: TGN and TGAT.
    e. Support node cache for accelerating distributed feature lookup.

  • Enhancements for Dev:
    a. Add a devcontainer.json for users to easily set up a developing environment using vscode.

  • Other enhancements and bug fixes:
    a. 'helm install graphscope/graphscope' would now launch all pods upon the start.
    b. Allowing more fine-grained control over Groot deployment
    c. Fixed an error of 'lgraph' that could cause the building failure in M1 Mac

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.21.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.21.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.21.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.21.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.21.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.21.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.21.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

Read more

Contributors

yecol, LiSu, and 18 other contributors
Assets 6
Loading

Release v0.20.0

16 Mar 07:07
@lidongze0629 lidongze0629
v0.20.0
8c98bc5
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.20.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.20.0 release. This release contains many important features and improvements to the persistent graph store Groot, including load balance and failover mechanism. In addition, Graph Learning Engine (GLE) now supports the Aliyun ACK cluster.

We highlight the following improvements included in this release:

1. Enhancements for GLE:

  • GLE could be launched and used to training on ACK cluster.

2. Enhancements for Groot:

  • Chart is refactored and has more customizable fields.
  • LoadBalancher service is supported.
  • Optimization of failover mechanism.

3. Enhancement for images:

  • We release the first dev image (graphscope-dev) for arm64 platform.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.20.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.20.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.20.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.20.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.20.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.20.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.20.0

What's Changed

Full Changelog: v0.19.0...v0.20.0

Contributors

yecol, LiSu, and 8 other contributors
Assets 2
Loading

Release v0.19.0

18 Jan 07:27
@lidongze0629 lidongze0629
v0.19.0
22a3b42
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release v0.19.0

We are glad to present the release of GraphScope 0.19.0, which brings a number of new features as well as performance and stability improvements to GraphScope. Specially, users can try GraphScope in python3.10 and python3.11 environments of Linux and MacOS by pip3 install graphscope; This release also optimizes the memory usage for loading graphs, which allows users to load larger graphs on their machines. In addition, we improve Graph Interactive Engine (GIE) via a couple of bug fixes and improvements.

We highlight the following improvements included in this release:

1. Optimized memory usage for loading graphs:

  • Improves memory usage during loading and projecting graphs, especially for graphs with string as OID type.
  • Improves the performance of project_to_simple.

2. Bug fix and enhancements for GIE:

  • Fix a bug in Pegasus that may block the execution of interactive queries in certain case.
  • Fix a bug that fails to configure the server number for the GIE engine.
  • Fix multiple dependencies in both Java and Rust codes.
  • Resolve a issue that JNA could not map bool correctly.

3. Enhancements for GAE Java:

  • Pack JNI lib into the grape-runtime jar and fix some dependency issues.

4. Standalone deployment:

  • Components now have their own dedicated image, and user could configure a subset of engines to use when launching session.

5. Other bug fixes:

  • Fix helm deployment could not delete role and rolebindings when uninstalling GraphScope.
  • Fix compile error under the newer toolchain and arrow 10
  • Rename maxgraph to graphscope and other refactors
  • Cleanup overlapping rust dependencies

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.19.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.19.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.19.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.19.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.19.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.19.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.19.0

What's Changed

Read more

Contributors

yecol, LiSu, and 11 other contributors
Assets 3
Loading