Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Integrate groot into flex architecture #3566

Merged
merged 17 commits into from
Mar 15, 2024

Conversation

lidongze0629
Copy link
Collaborator

@lidongze0629 lidongze0629 commented Feb 21, 2024

What do these changes do?

  • Flex Coordinator: Support Groot storage, including operating graph、schema、and dataloading job.
  • gsctl: Add create vtype/etype create schema delete vtype/etype command
gsctl create vtype/etype -f /path/to/file
gsctl delete vtype person
gsctl delete etype knows -s person -d person
gsctl create schema -f /home/graphscope/alibaba/test/groot/schema.yaml
  • Add batch interface for updating vertex/edge properties of Groot
def update_vertex_properties_batch(self, vertices: list):
        request = to_write_requests_pb("VERTEX", vertices, write_service_pb2.UPDATE)
        return self._conn.batch_write(request)

def update_edge_properties_batch(self, edges: list):
        request = to_write_requests_pb("EDGE", edges, write_service_pb2.UPDATE)
        return self._conn.batch_write(request)
  • Integrate Coordinator into Groot image, now we can build Flex Groot(Insight image through gsctl flexbuild command
gsctl flexbuild insight --app docker

Doc: Documentation will be added later when the final format of gsctl is determined.

@lidongze0629 lidongze0629 force-pushed the flex_groot branch 2 times, most recently from 39ea7f8 to 186b928 Compare February 28, 2024 08:23
@lidongze0629 lidongze0629 force-pushed the flex_groot branch 2 times, most recently from 5865940 to f2c4e74 Compare March 8, 2024 09:56
@lidongze0629 lidongze0629 changed the title [WIP] Integrate groot into flex architecture feat: Integrate groot into flex architecture Mar 11, 2024
@lidongze0629 lidongze0629 force-pushed the flex_groot branch 3 times, most recently from 7aa90e1 to 0b94a91 Compare March 14, 2024 08:46
siyuan0322
siyuan0322 previously approved these changes Mar 14, 2024
@codecov-commenter
Copy link

codecov-commenter commented Mar 14, 2024

Codecov Report

Attention: Patch coverage is 0.17376% with 1149 lines in your changes are missing coverage. Please review.

Project coverage is 25.96%. Comparing base (1ebbde6) to head (a05bacd).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3566      +/-   ##
==========================================
- Coverage   27.76%   25.96%   -1.81%     
==========================================
  Files         178      187       +9     
  Lines       16245    17379    +1134     
==========================================
+ Hits         4511     4513       +2     
- Misses      11734    12866    +1132     
Files Coverage Δ
python/graphscope/gsctl/commands/common.py 0.00% <ø> (ø)
python/graphscope/gsctl/commands/interactive.py 0.00% <ø> (ø)
python/graphscope/flex/rest/api/__init__.py 0.00% <0.00%> (ø)
python/graphscope/gsctl/impl/job.py 0.00% <0.00%> (ø)
...hon/graphscope/flex/rest/models/deployment_info.py 0.00% <0.00%> (ø)
python/graphscope/gsctl/commands/__init__.py 0.00% <0.00%> (ø)
python/graphscope/client/connection.py 36.92% <33.33%> (-0.18%) ⬇️
python/graphscope/flex/rest/models/__init__.py 0.00% <0.00%> (ø)
python/graphscope/flex/rest/__init__.py 0.00% <0.00%> (ø)
python/graphscope/gsctl/impl/utils.py 0.00% <0.00%> (ø)
... and 15 more

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ebbde6...a05bacd. Read the comment docs.

@sighingnow sighingnow merged commit e283fa4 into alibaba:main Mar 15, 2024
71 of 73 checks passed
@sighingnow sighingnow deleted the flex_groot branch March 15, 2024 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants