Skip to content

Commit

Permalink
Revert "[serve] Replace Ray Client with Ray Job Submission for `serve… (
Browse files Browse the repository at this point in the history
ray-project#32976)

- Previously, if connecting to a local cluster, `serve run` will directly call `ray.init()` and `serve.run`
- With the change in ray-project#30913, even when connecting to a local cluster, `serve run` would submit a Ray Job to the local cluster and then call `ray.init()` and `serve.run()` inside that job. This caused the issue ray-project#32881, where if a user ran `serve run` inside a Job (or any other process with a runtime environment), they should expect to have access to the dependencies installed in that runtime environment. However since `serve run` goes through another layer of Job submission, they don't.

Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
  • Loading branch information
zcin committed Mar 3, 2023
1 parent 4032d02 commit ef2e1cd
Show file tree
Hide file tree
Showing 5 changed files with 130 additions and 230 deletions.
30 changes: 14 additions & 16 deletions doc/source/ray-contribute/docs.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -194,11 +194,7 @@
"cell_type": "code",
"execution_count": null,
"id": "ba88d95f",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"metadata": {},
"outputs": [],
"source": [
"# __function_api_start__\n",
Expand Down Expand Up @@ -329,10 +325,7 @@
"metadata": {
"tags": [
"hide-cell"
],
"vscode": {
"languageId": "python"
}
]
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -393,11 +386,7 @@
"cell_type": "code",
"execution_count": null,
"id": "8412103e",
"metadata": {
"vscode": {
"languageId": "python"
}
},
"metadata": {},
"outputs": [],
"source": [
"num_workers = 8\n",
Expand Down Expand Up @@ -441,11 +430,20 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "ray",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.8.15"
},
"vscode": {
"interpreter": {
"hash": "1be35923f02ee4768c0de6a4500ac71c66a0bb6d3c7909b7f12beb33d1dc2c00"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
}
12 changes: 4 additions & 8 deletions doc/source/serve/dev-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,6 @@ serve run local_dev:graph

The `serve run` command blocks the terminal and can be canceled with Ctrl-C. Typically, `serve run` should not be run simultaneously from multiple terminals, unless each `serve run` is targeting a separate running Ray cluster.

:::{note}
If you already have a local Ray Cluster running before executing `serve run`, make sure that the path to your Serve app is accessible from the working directory in which you started the Ray Cluster using `ray start --head`. Otherwise, you can pass in `app-dir` or `working-dir` when executing `serve run`. See the documentation for [serve run](serve_cli.html#serve-run) for more details.
:::

Now that Serve is running, we can send HTTP requests to the application.
For simplicity, we'll just use the `curl` command to send requests from another terminal.

Expand All @@ -85,17 +81,17 @@ Note that rerunning `serve run` will redeploy all deployments. To prevent redepl

## Testing on a remote cluster

To test on a remote cluster, you'll use `serve run` again, but this time you'll pass in an `--address` argument to specify the address of the Ray cluster to connect to. For remote clusters, this address has the form `http://<head-node-ip-address>:8265` and will be passed to Ray Job Submission; see [Ray Jobs](jobs-overview) for more information.
To test on a remote cluster, you'll use `serve run` again, but this time you'll pass in an `--address` argument to specify the address of the Ray cluster to connect to. For remote clusters, this address has the form `ray://<head-node-ip-address>:10001`; see [Ray Client](ray-client-ref) for more information.

When making the transition from your local machine to a remote cluster, you'll need to make sure your cluster has a similar environment to your local machine--files, environment variables, and Python packages, for example.

Let's see a simple example that just packages the code. Run the following command on your local machine, with your remote cluster head node IP address substituted for `<head-node-ip-address>` in the command:

```bash
serve run --address=http://<head-node-ip-address>:8265 --working_dir="./project/src" local_dev:graph
serve run --address=ray://<head-node-ip-address>:10001 --working_dir="./project/src" local_dev:graph
```

This will upload the `working_dir` directory to the remote cluster and run your Serve application as a Ray Job on the remote cluster. Here, the local directory specified by `working_dir` must contain `local_dev.py` so that it can be uploaded to the cluster and imported by Ray Serve.
This will connect to the remote cluster via Ray Client, upload the `working_dir` directory, and run your serve application. Here, the local directory specified by `working_dir` must contain `local_dev.py` so that it can be uploaded to the cluster and imported by Ray Serve.

Once this is up and running, we can send requests to the application:

Expand All @@ -107,7 +103,7 @@ curl -X PUT http://<head-node-ip-address>:8000/?name=Ray
For more complex dependencies, including files outside the working directory, environment variables, and Python packages, you can use {ref}`Runtime Environments<runtime-environments>`. Here is an example using the --runtime-env-json argument:

```bash
serve run --address=http://<head-node-ip-address>:8265 --runtime-env-json='{"env_vars": {"MY_ENV_VAR": "my-value"}, "working_dir": "./project/src", "pip": ["requests", "chess"]}' local_dev:graph
serve run --address=ray://<head-node-ip-address>:10001 --runtime-env-json='{"env_vars": {"MY_ENV_VAR": "my-value"}, "working_dir": "./project/src", "pip": ["requests", "chess"]}' local_dev:graph
```

You can also specify the `runtime_env` via a YAML file; see [serve run](serve_cli.html#serve-run) for details.
Expand Down
124 changes: 0 additions & 124 deletions python/ray/serve/run_script.py

This file was deleted.

0 comments on commit ef2e1cd

Please sign in to comment.