Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: EngineArgs missing value type for lora_dtype #5397

Closed
c3-ali opened this issue Jun 10, 2024 · 0 comments · Fixed by #5398 · May be fixed by #5428
Closed

[Bug]: EngineArgs missing value type for lora_dtype #5397

c3-ali opened this issue Jun 10, 2024 · 0 comments · Fixed by #5398 · May be fixed by #5428
Labels
bug Something isn't working

Comments

@c3-ali
Copy link
Contributor

c3-ali commented Jun 10, 2024

Your current environment

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] torch==2.3.0
[pip3] transformers==4.41.1
[pip3] triton==2.3.0
[pip3] vllm-nccl-cu12==2.18.1.0.4.0
[conda] numpy                     1.22.4           py39hc58783e_0    conda-forge
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] torch                     2.3.0                    pypi_0    pypi
[conda] transformers              4.41.1                   pypi_0    pypi
[conda] triton                    2.3.0                    pypi_0    pypi
[conda] vllm-nccl-cu12            2.18.1.0.4.0             pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.4.2

🐛 Describe the bug

import vllm
vllm.EngineArgs(model='a', lora_dtype='float16')

raises

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[40], line 1
----> 1 ea = vllm.EngineArgs(model='a', lora_dtype='float16')

TypeError: __init__() got an unexpected keyword argument 'lora_dtype'

Fix

Modify lora_dtype = 'auto' to lora_dtype: str = 'auto' in arg_utils.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant