-
-
Notifications
You must be signed in to change notification settings - Fork 11.5k
Open
Labels
usageHow to use vllmHow to use vllm
Description
Your current environment
I run this to install vllm with the latest code. But, the installed vllm doesn't include the code I need.
I check the siglip.py file, it's modified 4 days ago.
But in the vllm installed, it doesn't contain this commit! https://github.com/vllm-project/vllm/pull/27566/files#diff-ca771e5a262cbf32fb481c518bea41d0e341414e021d6542e421abb98cceec61
why is this?
I use this command.
pip install -U vllm \
--pre \
--extra-index-url https://wheels.vllm.ai/nightly```
`pip install -U vllm \
--pre \
--extra-index-url https://wheels.vllm.ai/nightly
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://bytedpypi.byted.org/simple, https://bytedpypi.byted.org/simple, https://wheels.vllm.ai/nightly
Requirement already satisfied: vllm in /home/alice/.local/lib/python3.10/site-packages (0.11.0)
Collecting vllm
Downloading https://wheels.vllm.ai/nightly/vllm-0.11.1rc6.dev16%2Bg933cdea44.cu129-cp38-abi3-manylinux1_x86_64.whl (479.0 MB)
━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.8/479.0 MB 575.3 kB/s eta 0:13:22`
### How would you like to use vllm
I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.
### Before submitting a new issue...
- [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.
Metadata
Metadata
Assignees
Labels
usageHow to use vllmHow to use vllm