Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Make gsctl to be more like a database design for Groot storage #3651

Merged
merged 3 commits into from Mar 26, 2024

Conversation

lidongze0629
Copy link
Collaborator

@lidongze0629 lidongze0629 commented Mar 25, 2024

What do these changes do?

gsctl

graphscope@host:/work/python$ gsctl

    ______                 __   _____
   / ____/________ _____  / /_ / ___/_________  ____  ___
  / / __/ ___/ __ `/ __ \/ __ \\__ \/ ___/ __ \/ __ \/ _ \
 / /_/ / /  / /_/ / /_/ / / / /__/ / /__/ /_/ / /_/ /  __/
 \____/_/   \__,_/ .___/_/ /_/____/\___/\____/ .___/\___/
                /_/                         /_/

Currently, gsctl hasn't connect to any service.

you can use gsctl as an utility script.
Or you can connect to a launched GraphScopoe service by `gsctl connect --coordinator-endpoint <address>`.
See more detailed information at https://graphscope.io/docs/utilities/gs.

Usage: gsctl [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  close         Disconnect from coordinator
  connect       Connect to a launched coordinator
  flexbuild     Build docker image for Interactive, Insight product.
  install-deps  Install dependencies for building GraphScope.
  make          Build executive binaries of COMPONENT.
  make-image    Make docker images from source code for deployment.
  test          Trigger tests on built artifacts.

connect to Coordinator(serving with Groot mode)

graphscope@host:/work/python$ gsctl connect --coordinator-endpoint http://127.0.0.1:8081
[SUCCESS] Connected to http://127.0.0.1:8081, coordinator is serving with GRAPHSCOPE_INSIGHT mode.

Try 'gsctl --help' for help.

graphscope@host:/work/python$ gsctl
Usage: gsctl [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  close    Disconnect from coordinator
  connect  Connect to a launched coordinator
  create   Create vertex/edge type, data source, loader job from file
  delete   Delete vertex/edge type, data source, loader job by identifier
  desc     Show details of job status by identifier
  ls       Display schema, stored procedure, and job information

gsctl ls

graphscope@host:/work/python$ gsctl ls
schema
├── vertex types
│   ├── person
│   │   ├── Property(name: id, type: LONG, is_primary_key: True)
│   │   ├── Property(name: name, type: STRING, is_primary_key: False)
│   │   └── Property(name: age, type: LONG, is_primary_key: False)
│   └── software
│       ├── Property(name: id, type: LONG, is_primary_key: True)
│       ├── Property(name: name, type: STRING, is_primary_key: False)
│       └── Property(name: lang, type: STRING, is_primary_key: False)
└── edge types
    ├── (person) -[created]-> (software)
    │   ├── Property(name: edge_id, type: LONG, is_primary_key: False)
    │   └── Property(name: weight, type: DOUBLE, is_primary_key: False)
    └── (person) -[knows]-> (person)
        ├── Property(name: edge_id, type: LONG, is_primary_key: False)
        └── Property(name: weight, type: DOUBLE, is_primary_key: False)

data sources
├── vertex mappings
│   ├── person(input: /home/graphscope/.graphscope/gs/dataset/person.csv)
│   │   ├── Property(name: id) -> DataSourceColumn(index: 0)
│   │   ├── Property(name: name) -> DataSourceColumn(index: 1)
│   │   └── Property(name: age) -> DataSourceColumn(index: 2)
│   └── software(input: /home/graphscope/.graphscope/gs/dataset/software.csv)
│       ├── Property(name: id) -> DataSourceColumn(index: 0)
│       ├── Property(name: name) -> DataSourceColumn(index: 1)
│       └── Property(name: lang) -> DataSourceColumn(index: 2)
└── edge mappings
    ├── (person) -[knows]-> (person)(input: /home/graphscope/.graphscope/gs/dataset/person_knows_person.csv)
    │   ├── SourceVertexPrimaryKey(name: id) -> DataSourceColumn(index: 0)
    │   ├── DestinationVertexPrimaryKey(name: id) -> DataSourceColumn(index: 1)
    │   └── Property(name: weight) -> DataSourceColumn(index: 2)
    └── (person) -[created]-> (software)(input: /home/graphscope/.graphscope/gs/dataset/person_created_software.csv)
        ├── SourceVertexPrimaryKey(name: id) -> DataSourceColumn(index: 0)
        ├── DestinationVertexPrimaryKey(name: id) -> DataSourceColumn(index: 1)
        └── Property(name: weight) -> DataSourceColumn(index: 2)

stored procedure

job
├── Job(identifier: JOB-RTTGLHRKQKGGICJK, type: Data Import, status: FAILED, start time: 2024-03-25 15:57:39, end time: 2024-03-25 15:57:39)
├── Job(identifier: JOB-SJPKAZFLIXHNKCAN, type: Data Import, status: FAILED, start time: 2024-03-25 16:03:51, end time: 2024-03-25 16:03:51)
└── Job(identifier: JOB-PEZFBOLGPHYZKWPK, type: Data Import, status: SUCCESS, start time: 2024-03-25 16:04:49, end time: 2024-03-25 16:04:49)

schema、data source operation

graphscope@host:/work/python$ gsctl delete vertex_type software
[SUCCESS] Delete vertex type software successfully.

graphscope@host:/work/python$ gsctl delete vertex_type person
[SUCCESS] Delete vertex type person successfully.

graphscope@host:/work/python$ gsctl create schema -f /home/graphscope/alibaba/test/groot/schema.yaml
[SUCCESS] Import schema successfully.

graphscope@host:/work/python$ gsctl create datasource -f /home/graphscope/alibaba/test/groot/datasource.yaml
[SUCCESS] Bind data source successfully.

job operation

graphscope@host:/work/python$ gsctl create loaderjob -f /home/graphscope/alibaba/test/groot/jobconfig.yaml
[SUCCESS] Create job SCHEDULER-ZPABGJMPYCKKGNZI successfully.

graphscope@host:/work/python$ gsctl desc job JOB-ZPABGJMPYCKKGNZI
detail:
  graph_name: default-graph
  label: person,software,person_knows_person,person_created_software
end_time: '2024-03-25 15:57:39'
job_id: JOB-ZPABGJMPYCKKGNZI
log: Vertex type software not exists
start_time: '2024-03-25 15:57:39'
status: FAILED

gsctl close

graphscope@host:/work/python$ gsctl close
Disconnecting from the context: {'name': 'context_vUCpvqPZ', 'flex': 'GRAPHSCOPE_INSIGHT', 'coordinator_endpoint': 'http://127.0.0.1:8081', 'context': 'global', 'timestamp': 1711356880.1457205}
[SUCCESS] Coordinator service disconnected.

@sighingnow sighingnow merged commit 421c175 into alibaba:main Mar 26, 2024
68 of 70 checks passed
@sighingnow sighingnow deleted the flex_groot branch March 26, 2024 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants