-
Notifications
You must be signed in to change notification settings - Fork 485
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump vLLM version to 0.3.0
#579
Conversation
LLMEngine.tokenizer
uses a new class now: TokenizerGroup
LLMEngine.tokenizer
uses a new class now: TokenizerGroup
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(you should also be able to remove this line as vLLM 0.3.0 did migrate to pydantic 2.0)
pyproject.toml
Outdated
@@ -59,7 +59,7 @@ test = [ | |||
"huggingface_hub" | |||
] | |||
serve = [ | |||
"vllm>=0.2.6", | |||
"vllm>=0.2.8", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vLLM skipped 0.2.8
"vllm>=0.2.8", | |
"vllm>=0.3.0", |
Let's add |
01d7de5
to
bbbc22c
Compare
@lapp0 I made a few changes on your branch. Can you confirm that it works on your end, I don't have access to my desktop today. |
bbbc22c
to
a22d47d
Compare
@remic33 yes, I will smoke test in a few hours and report the results. |
Change-set looks good after a simple smoke test. On this branch with
|
All good then, merging |
LLMEngine.tokenizer
uses a new class now:TokenizerGroup
pydantic
Fixes #576
Fixes #504
vLLM
main
is incompatible with outlines. Do not merge untilvllm==0.2.8
is released, leaving as draft until then.