Add ray[default] to wget to run distributed inference out of box #11265
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
part of #11137
Why do we need such PR?
ray[default]
package provides more capabilities like submission api and dashboard which is essential to run vLLM distributed inference in cluster environmentwget
is used in kubeRay prob.https://github.com/ray-project/kuberay/blob/e595ee4c6297fb6b385421f7ca34fbd7c1c0b49f/ray-operator/controllers/ray/common/pod.go#L253 and https://github.com/ray-project/kuberay/blob/e595ee4c6297fb6b385421f7ca34fbd7c1c0b49f/ray-operator/controllers/ray/utils/constant.go#L178. This can be definitely changed tocurl
but I feel adding a new package won't take that many room here.wget
andray[default]
won't be risky. I notice some other files already include this change.vllm/requirements-tpu.txt
Line 11 in f9ecbb1
vllm/Dockerfile.arm
Line 11 in f9ecbb1
FIX