Skip to content

Commit

Permalink
Revert "[serve] Replace Ray Client with Ray Job Submission for `serve… (
Browse files Browse the repository at this point in the history
ray-project#32976)

- Previously, if connecting to a local cluster, `serve run` will directly call `ray.init()` and `serve.run`
- With the change in ray-project#30913, even when connecting to a local cluster, `serve run` would submit a Ray Job to the local cluster and then call `ray.init()` and `serve.run()` inside that job. This caused the issue ray-project#32881, where if a user ran `serve run` inside a Job (or any other process with a runtime environment), they should expect to have access to the dependencies installed in that runtime environment. However since `serve run` goes through another layer of Job submission, they don't.

Signed-off-by: Jack He <jackhe2345@gmail.com>
  • Loading branch information
zcin authored and ProjectsByJackHe committed May 4, 2023
1 parent 101de31 commit 7cea4a8
Show file tree
Hide file tree
Showing 4 changed files with 116 additions and 214 deletions.
12 changes: 4 additions & 8 deletions doc/source/serve/dev-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,6 @@ serve run local_dev:graph

The `serve run` command blocks the terminal and can be canceled with Ctrl-C. Typically, `serve run` should not be run simultaneously from multiple terminals, unless each `serve run` is targeting a separate running Ray cluster.

:::{note}
If you already have a local Ray Cluster running before executing `serve run`, make sure that the path to your Serve app is accessible from the working directory in which you started the Ray Cluster using `ray start --head`. Otherwise, you can pass in `app-dir` or `working-dir` when executing `serve run`. See the documentation for [serve run](serve_cli.html#serve-run) for more details.
:::

Now that Serve is running, we can send HTTP requests to the application.
For simplicity, we'll just use the `curl` command to send requests from another terminal.

Expand All @@ -85,17 +81,17 @@ Note that rerunning `serve run` will redeploy all deployments. To prevent redepl

## Testing on a remote cluster

To test on a remote cluster, you'll use `serve run` again, but this time you'll pass in an `--address` argument to specify the address of the Ray cluster to connect to. For remote clusters, this address has the form `http://<head-node-ip-address>:8265` and will be passed to Ray Job Submission; see [Ray Jobs](jobs-overview) for more information.
To test on a remote cluster, you'll use `serve run` again, but this time you'll pass in an `--address` argument to specify the address of the Ray cluster to connect to. For remote clusters, this address has the form `ray://<head-node-ip-address>:10001`; see [Ray Client](ray-client-ref) for more information.

When making the transition from your local machine to a remote cluster, you'll need to make sure your cluster has a similar environment to your local machine--files, environment variables, and Python packages, for example.

Let's see a simple example that just packages the code. Run the following command on your local machine, with your remote cluster head node IP address substituted for `<head-node-ip-address>` in the command:

```bash
serve run --address=http://<head-node-ip-address>:8265 --working_dir="./project/src" local_dev:graph
serve run --address=ray://<head-node-ip-address>:10001 --working_dir="./project/src" local_dev:graph
```

This will upload the `working_dir` directory to the remote cluster and run your Serve application as a Ray Job on the remote cluster. Here, the local directory specified by `working_dir` must contain `local_dev.py` so that it can be uploaded to the cluster and imported by Ray Serve.
This will connect to the remote cluster via Ray Client, upload the `working_dir` directory, and run your serve application. Here, the local directory specified by `working_dir` must contain `local_dev.py` so that it can be uploaded to the cluster and imported by Ray Serve.

Once this is up and running, we can send requests to the application:

Expand All @@ -107,7 +103,7 @@ curl -X PUT http://<head-node-ip-address>:8000/?name=Ray
For more complex dependencies, including files outside the working directory, environment variables, and Python packages, you can use {ref}`Runtime Environments<runtime-environments>`. Here is an example using the --runtime-env-json argument:

```bash
serve run --address=http://<head-node-ip-address>:8265 --runtime-env-json='{"env_vars": {"MY_ENV_VAR": "my-value"}, "working_dir": "./project/src", "pip": ["requests", "chess"]}' local_dev:graph
serve run --address=ray://<head-node-ip-address>:10001 --runtime-env-json='{"env_vars": {"MY_ENV_VAR": "my-value"}, "working_dir": "./project/src", "pip": ["requests", "chess"]}' local_dev:graph
```

You can also specify the `runtime_env` via a YAML file; see [serve run](serve_cli.html#serve-run) for details.
Expand Down
124 changes: 0 additions & 124 deletions python/ray/serve/run_script.py

This file was deleted.

0 comments on commit 7cea4a8

Please sign in to comment.