[Core][Doc] Default to multiprocessing for single-node distributed case #5230
Conversation
Also update docs to reflect support for the multiprocessing distributed executor. Resolves vllm-project#4955 Resolves vllm-project#5221
|
If we detect we are in a Ray placement group ( |
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
|
Would it possible to also make ray optional or as an extra when installing vllm as part of this PR? Currently it's being installed by default as part of the requirements for vllm so it'll always be present? |
|
@vrdn-23 I had considered this and actually have a change for it in a branch since this is what we do in our own builds. But right now there are other dependencies included that could be considered optional as well, such as Would be good to get thoughts from others on this too. |
|
with all these dependencies (not only affect ray, also other components that may cause any issue), I'd suggest to open another PR from the branch that is suggested by @njhill alongside another GH issue to track the effort. |
|
+1 I'm in favor of optional dependencies and trimming down our dependencies in general. Please open PRs! |
|
I think these are actual failures https://buildkite.com/vllm/ci-aws/builds/646#018ff31c-991c-439e-b417-efee7c59e5c0 |
…se (vllm-project#5230) Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
|
+1 |
…se (vllm-project#5230) Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
Also update docs to reflect support for the multiprocessing distributed executor.
Resolves #4955
Resolves #5221