Skip to content

Commit

Permalink
Update doc for gsctl
Browse files Browse the repository at this point in the history
  • Loading branch information
@lidongze0629
lidongze0629 committed Mar 22, 2024
1 parent 68afa78 commit 7908703
Show file tree
Hide file tree
Showing 7 changed files with 174 additions and 130 deletions.
44 changes: 31 additions & 13 deletions docs/utilities/gs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,26 @@

Since it is shipped with python package `graphscope-client`, the `gsctl` command will be available in your terminal after installing GraphScope:
```bash
pip install graphscope-client
$ pip3 install graphscope-client
```

In some cases, such as development on `gsctl`, you may want to build it from source.
In some cases, such as development on `gsctl`, you may want to build it from source.
To do this, navigate to the directory where the source code is located and run the following command:

```bash
cd REPO_HOME/python
# If you want to develop gsctl,
# please note the entry point is located on:
$ cd REPO_HOME/python
# If you want to develop gsctl,
# please note the entry point is located on:
# /python/graphscope/gsctl/gsctl.py
pip install --editable .
$ pip3 install --editable .
```
This will install `gsctl` in an editable mode, which means that any changes you make to the source code will be reflected in the installed version of `gsctl`.

## Commands

With `gsctl`, you can do the following things. Always remember to use `--help` on a command to get more information.

The `gsctl` command-line utility supports two modes of operation: utility scripts and client/server mode. You can switch between these modes using the
`gsctl connect` and `gsctl close` commands.
The `gsctl` command-line utility supports two modes of operation: utility scripts and client/server mode. You can switch between these modes using the `gsctl connect` and `gsctl close` commands.

### Utility Scripts

Expand All @@ -39,12 +38,31 @@ Default, the `gsctl` provide helper functions and utilities that can be run usin
- `gsctl test`, trigger test suites.
- `gsctl connect`, connect to the launched coordinator by ~/.gs/config.
- `gsctl close`, Close the connection from the coordinator.

- `gsctl flexbuild`, Build docker image for Interactive, Insight product.

### Client/Server Mode

To switch to the client/server mode, use the `gsctl connect` command. This command connects gsctl to a launched coordinator using the configuration file located at ~/.gsconfig.
Once connected, you can use `gsctl` to communicate with the coordinator and send commands that will be executed on the coordinator side.
To switch to the client/server mode, use the `gsctl connect` command. By default, this command connects gsctl to a launched coordinator using the configuration file located at `${HOME}/.gsctl`; If `--coordinator-endpoint` param is specified, it will treat it as the current context and override the configuration file.

Once connected, you can use `gsctl` to communicate with the coordinator which serves the specific Flex product behind it.

#### Change scope

In `gsctl`, you can run commands on a global scope or a local scope. When you connect to a coordinator, you are in the global scope. To change to local scope of a graph, run the `gsctl use GRAPH <graph_identifier>` command. You can find the graph identifier with `gsctl ls` command.

```bash
$ gsctl use GRAPH modern_graph
Using GRAPH modern_graph
```

To switch back to the global scope. run `gsctl use GLOBAL` command.

```bash
$ gsctl use GLOBAL
Using GLOBAL
```
Different scopes have different commands. Always remember to use `--help` on a command to get more information.

#### Close the connection

To disconnect from the coordinator and switch back to the utility scripts mode, you can use the `gsctl close` command. This command closes the connection from the coordinator
and allows you to use `gsctl` as a standalone utility again.
To disconnect from the coordinator and switch back to the utility scripts mode, you can use the `gsctl close` command. This command closes the connection from the coordinator and allows you to use `gsctl` as a standalone utility again.
33 changes: 18 additions & 15 deletions python/graphscope/gsctl/commands/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@
from graphscope.gsctl.config import load_gs_config
from graphscope.gsctl.config import logo
from graphscope.gsctl.impl import connect_coordinator
from graphscope.gsctl.utils import err
from graphscope.gsctl.utils import info


def get_command_collection(context: Context):
Expand All @@ -40,13 +42,14 @@ def get_command_collection(context: Context):
# build graphscope locally.
if context is None:
if len(sys.argv) == 1:
click.secho(logo, fg="green", bold=True)
info(logo, fg="green", bold=True)
click.secho("Currently, gsctl hasn't connect to any service.", fg="yellow")
message = """
Currently, gsctl hasn't connect to any service, you can use gsctl as an utility script.
you can use gsctl as an utility script.
Or you can connect to a launched GraphScopoe service by `gsctl connect --coordinator-endpoint <address>`.
See more detailed informations at https://graphscope.io/docs/utilities/gs.
See more detailed information at https://graphscope.io/docs/utilities/gs.
"""
click.secho(message, fg="green")
info(message)
return commands

if context.is_expired():
Expand All @@ -55,14 +58,14 @@ def get_command_collection(context: Context):
response = connect_coordinator(context.coordinator_endpoint)
solution = response.solution
except Exception as e:
click.secho(
err(
"Failed to connect to coordinator at {0}: {1}".format(
context.coordinator_endpoint, str(e)
),
fg="red",
)
)
info(
"Please check the availability of the service, fall back to the default commands."
)
click.secho("Please check the availability of the service.", fg="red")
click.secho("Fall back to the default commands.", fg="red")
return commands
else:
# check consistency
Expand All @@ -76,14 +79,14 @@ def get_command_collection(context: Context):

if context.flex == "INTERACTIVE":
if context.context == "global":
if len(sys.argv) == 1:
message = f"Using global, to change to a specific graph context, run `gsctl use graph <graph_identifier>`.\n"
click.secho(message, fg="green")
info("Using GLOBAL.", fg="green", bold=True)
info(
"Run `gsctl use GRAPH <graph_identifier>` to switch to a specific graph context.\n"
)
commands = click.CommandCollection(sources=[common, interactive])
else:
if len(sys.argv) == 1:
message = f"Using graph {context.context}, to switch back to the global, run `gsctl use global`.\n"
click.secho(message, fg="green")
info(f"Using GRAPH {context.context}.", fg="green", bold=True)
info("Run `gsctl use GLOBAL` to switch back to GLOBAL context.\n")
commands = click.CommandCollection(sources=[common, interactive_graph])
elif context.flex == "GRAPHSCOPE_INSIGHT":
if context.context == "global":
Expand Down
24 changes: 14 additions & 10 deletions python/graphscope/gsctl/commands/common.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@
from graphscope.gsctl.config import load_gs_config
from graphscope.gsctl.impl import connect_coordinator
from graphscope.gsctl.impl import disconnect_coordinator
from graphscope.gsctl.utils import err
from graphscope.gsctl.utils import info
from graphscope.gsctl.utils import succ


@click.group()
Expand All @@ -37,7 +40,7 @@ def cli():
@click.option(
"-c",
"--coordinator-endpoint",
help="Coordinator endpoint which gsctl connect to, e.g. http://127.0.0.1:9527",
help="Coordinator endpoint, e.g. http://127.0.0.1:9527",
)
def connect(coordinator_endpoint):
"""Connect to a launched coordinator
Expand All @@ -48,22 +51,21 @@ def connect(coordinator_endpoint):
if coordinator_endpoint is None:
context = get_current_context()
if context is None:
click.secho(
"No available context, try to connect to coordinator with --coordinator-endpoint",
fg="blue",
err(
"No available context found, try to connect by `gsctl conenct --coordinator-endpoint <addr>`."
)
return
coordinator_endpoint = context.coordinator_endpoint
# connect
try:
resp = connect_coordinator(coordinator_endpoint)
except Exception as e:
click.secho(f"Unable to connect to server: {str(e)}", fg="red")
err(f"Unable to connect to server: {str(e)}")
else:
click.secho(
f"Coordinator at {coordinator_endpoint} connected, {resp.solution} service is serving.",
fg="green",
succ(
f"Connected to {coordinator_endpoint}, coordinator is serving with {resp.solution} mode.\n"
)
info("Try 'gsctl --help' for help.")


@cli.command()
Expand All @@ -72,10 +74,12 @@ def close():
try:
context = disconnect_coordinator()
except Exception as e:
click.secho(f"Disconnect to server failed: {str(e)}", fg="red")
err(f"Disconnect from coordinator failed: {str(e)}")
else:
if context is not None:
click.secho(f"Coordinator disconnected: {context.to_dict()}.", fg="green")
info(f"Disconnecting from the context: {context.to_dict()}")
succ("Coordinator service disconnected.\n")
info("Try 'gsctl --help' for help.")


if __name__ == "__main__":
Expand Down
106 changes: 52 additions & 54 deletions python/graphscope/gsctl/commands/interactive/glob.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,11 @@
from graphscope.gsctl.impl import stop_service
from graphscope.gsctl.impl import switch_context
from graphscope.gsctl.utils import TreeDisplay
from graphscope.gsctl.utils import err
from graphscope.gsctl.utils import info
from graphscope.gsctl.utils import is_valid_file_path
from graphscope.gsctl.utils import read_yaml_file
from graphscope.gsctl.utils import succ
from graphscope.gsctl.utils import terminal_display


Expand All @@ -49,7 +52,7 @@ def create():

@cli.group()
def delete():
"""Delete a graph by id"""
"""Delete a graph by identifier"""
pass


Expand All @@ -59,42 +62,13 @@ def service():
pass


@cli.group()
def use():
"""Change to specific graph context"""
pass


@cli.command()
def ls(): # noqa: F811
"""Display all resources in database"""
tree = TreeDisplay()
try:
graphs = list_graphs()
for g in graphs:
# schema
tree.create_graph_node(g)
# get data source from job configuration
job_config = get_dataloading_config(g.name)
tree.create_datasource_node_for_interactive(g, job_config)
# stored procedure
procedures = list_procedures(g.name)
tree.create_procedure_node(g, procedures)
# job
jobs = list_jobs()
tree.create_job_node(g, jobs)
except Exception as e:
click.secho(f"Failed to list resources: {str(e)}", fg="red")
else:
message = "Using global, to change to a specific graph context, run `gsctl use graph <graph_identifier>`.\n"
click.secho(message, fg="green")
tree.show()


@use.command()
@click.argument(
"context", type=click.Choice(["GRAPH"], case_sensitive=False), required=True
)
@click.argument("GRAPH_IDENTIFIER", required=True)
def graph(graph_identifier): # noqa: F811
"""Change to specific graph context, see identifier with `ls` command"""
def use(context, graph_identifier):
"""Switch to GRAPH context, see identifier with `ls` command"""
try:
graphs = list_graphs()
graph_exist = False
Expand All @@ -108,9 +82,37 @@ def graph(graph_identifier): # noqa: F811
)
switch_context(graph_identifier)
except Exception as e:
click.secho(f"Failed to switch context: {str(e)}", fg="red")
err(f"Failed to switch context: {str(e)}")
else:
click.secho(f"Using graph {graph_identifier}.", fg="green")
click.secho(f"Using GRAPH {graph_identifier}", fg="green")


@cli.command()
@click.option("-l", is_flag=True, help="List sub resources recursively")
def ls(l): # noqa: F811, E741

Check notice on line 92 in python/graphscope/gsctl/commands/interactive/glob.py

View check run for this annotation

codefactor.io / CodeFactor

python/graphscope/gsctl/commands/interactive/glob.py#L92 

ambiguous variable name 'l' (E741)
"""Display graph resources in database"""
tree = TreeDisplay()
try:
graphs = list_graphs()
for g in graphs:
# schema
tree.create_graph_node(g, recursive=l)
if l:
# get data source from job configuration
job_config = get_dataloading_config(g.name)
tree.create_datasource_node_for_interactive(g, job_config)
# stored procedure
procedures = list_procedures(g.name)
tree.create_procedure_node(g, procedures)
# job
jobs = list_jobs()
tree.create_job_node(g, jobs)
except Exception as e:
err(f"Failed to list resources: {str(e)}")
else:
tree.show()
if not l:
info("Run `gsctl ls -l` to list all resources recursively.")


@create.command()
Expand All @@ -123,27 +125,27 @@ def graph(graph_identifier): # noqa: F811
def graph(filename): # noqa: F811
"""Create a new graph in database"""
if not is_valid_file_path(filename):
click.secho("Invalid file: {0}".format(filename), fg="red")
err(f"Invalid file: {filename}")
return
try:
graph = read_yaml_file(filename)
create_graph(graph)
except Exception as e:
click.secho(f"Failed to create graph: {str(e)}", fg="red")
err(f"Failed to create graph: {str(e)}")
else:
click.secho(f"Create graph {graph['name']} successfully.", fg="green")
succ(f"Create graph {graph['name']} successfully.")


@delete.command()
@click.argument("graph_identifier", required=True)
def graph(graph_identifier): # noqa: F811
"""Delete a graph by id, see graph identifier with `ls` command"""
"""Delete a graph by identifier, see graph identifier with `ls` command"""
try:
delete_graph_by_name(graph_identifier)
except Exception as e:
click.secho(f"Failed to delete graph {graph_identifier}: {str(e)}", fg="red")
err(f"Failed to delete graph {graph_identifier}: {str(e)}")
else:
click.secho(f"Delete graph {graph_identifier} successfully.", fg="green")
succ(f"Delete graph {graph_identifier} successfully.")


@service.command
Expand All @@ -152,9 +154,9 @@ def stop(): # noqa: F811
try:
stop_service()
except Exception as e:
click.secho(f"Failed to stop service: {str(e)}", fg="red")
err(f"Failed to stop service: {str(e)}")
else:
click.secho("Service stopped.", fg="green")
succ("Service stopped.")


@service.command
Expand All @@ -169,13 +171,9 @@ def start(graph_identifier): # noqa: F811
try:
start_service(graph_identifier)
except Exception as e:
click.secho(
f"Failed to start service on graph {graph_identifier}: {str(e)}", fg="red"
)
err(f"Failed to start service on graph {graph_identifier}: {str(e)}")
else:
click.secho(
f"Start service on graph {graph_identifier} successfully", fg="green"
)
succ(f"Start service on graph {graph_identifier} successfully")


@service.command
Expand All @@ -184,9 +182,9 @@ def restart(): # noqa: F811
try:
restart_service()
except Exception as e:
click.secho(f"Failed to restart service: {str(e)}", fg="red")
err(f"Failed to restart service: {str(e)}")
else:
click.secho("Service restarted.", fg="green")
succ("Service restarted.")


@service.command
Expand All @@ -209,7 +207,7 @@ def _construct_and_display_data(status):
try:
status = get_service_status()
except Exception as e:
click.secho(f"Failed to get service status: {str(e)}", fg="red")
err(f"Failed to get service status: {str(e)}")
else:
_construct_and_display_data(status)

Expand Down
Loading

0 comments on commit 7908703

Please sign in to comment.