Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an option to disable Ray when using a single GPU #23

Closed
WoosukKwon opened this issue Apr 2, 2023 · 0 comments · Fixed by #51
Closed

Add an option to disable Ray when using a single GPU #23

WoosukKwon opened this issue Apr 2, 2023 · 0 comments · Fixed by #51
Assignees
@zhuohan123

Comments

@WoosukKwon
Copy link
Collaborator

When working with a single GPU, Ray is not useful. Therefore, it would be beneficial to have an option to disable Ray in such scenarios.
@WoosukKwon WoosukKwon changed the title Add no-ray option for single gpu usage Add an option to disable Ray when using a single GPU Apr 2, 2023
@zhuohan123 zhuohan123 self-assigned this Apr 22, 2023
@zhuohan123 zhuohan123 mentioned this issue Apr 27, 2023
Merged
@shanshanpt shanshanpt mentioned this issue Nov 17, 2023
Closed
@junior-zsy junior-zsy mentioned this issue Nov 20, 2023
Closed
slyalin pushed a commit to slyalin/vllm that referenced this issue Apr 4, 2024
@ilya-lavrenov
Merge pull request vllm-project#23 from slyalin/paged_attention_in_op…
dbed638 
…envino

Use PagedAttentionExtension from OV without contrib dependency
z103cb referenced this issue in z103cb/opendatahub_vllm May 9, 2024
@tjohnson31415
ci/build/feat: bump vLLM libs to v0.4.2 and other deps in Dockerfile.…
c737a7a 
…ubi (opendatahub-io#23)

Changes:
- vLLM v0.4.2 was published today, update our build to use pre-built
libs from their wheel
- bump other dependencies in the image build (base UBI image, miniforge,
flash attention, grpcio-tools, accelerate)
- little cleanup to remove `PYTORCH_` args that are no longer used

---------

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
z103cb referenced this issue in z103cb/opendatahub_vllm May 9, 2024
@tjohnson31415 @z103cb
ci/build/feat: bump vLLM libs to v0.4.2 and other deps in Dockerfile.…
698088e 
…ubi (opendatahub-io#23)

Changes:
- vLLM v0.4.2 was published today, update our build to use pre-built
libs from their wheel
- bump other dependencies in the image build (base UBI image, miniforge,
flash attention, grpcio-tools, accelerate)
- little cleanup to remove `PYTORCH_` args that are no longer used

---------

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
z103cb referenced this issue in opendatahub-io/vllm May 9, 2024
@tjohnson31415 @z103cb
ci/build/feat: bump vLLM libs to v0.4.2 and other deps in Dockerfile.…
ed94d42 
…ubi (#23)

Changes:
- vLLM v0.4.2 was published today, update our build to use pre-built
libs from their wheel
- bump other dependencies in the image build (base UBI image, miniforge,
flash attention, grpcio-tools, accelerate)
- little cleanup to remove `PYTORCH_` args that are no longer used

---------

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
z103cb pushed a commit to dtrifiro/vllm that referenced this issue May 15, 2024
@openshift-merge-bot
Merge pull request vllm-project#23 from dtrifiro/use-vllm-nccl
3f5757e 
Dockerfile: use fixed vllm-provided nccl version
fxmarty pushed a commit to fxmarty/vllm-public that referenced this issue Jun 12, 2024
@charlifu
Merge pull request vllm-project#23 from ROCm/remove_duplicate_matvec
ae34618 
Removed HIP specific matvec logic that is duplicated from tuned_gemm.py and doesn't support bf16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhuohan123 zhuohan123

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add an option to launch cacheflow without ray vllm-project/vllm
2 participants
@zhuohan123 @WoosukKwon