-
-
Notifications
You must be signed in to change notification settings - Fork 10.5k
Remove V0 Encoder-Decoder Support #24907
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove V0 Encoder-Decoder Support #24907
Conversation
This pull request has merge conflicts that must be resolved before it can be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request restores support for Mllama4 and Skywork models. As part of this, it also removes a significant amount of code and tests related to a generic encoder-decoder model runner, effectively dropping support for models like BART, mBART, Donut, Florence2, and the original Mllama. While the changes are extensive, they appear to be mostly consistent with this goal. However, I've identified two critical bugs in an example script that will lead to NameError
exceptions at runtime due to variables being used after their definitions were removed.
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
848b568
to
c66902d
Compare
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
@WoosukKwon your PR is deleting Donut model from offline inference examples. Could you please let me know why Donut model is not longer supported while this PR has been merged recently #23229 ? |
@mfournioux We were planning to discontinue encoder-decoder model support (except Whisper) in the course of V0 deprecation. I think the PR was merged without the lack of this information. |
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Many thanks for your reply, why only Whisper model is kept and others like Donut are not longer supported regarding this PR ? It means Donut model is not longer supported for next vllm version ? |
Remove V0 encoder decoder model runner.
Also, this PR deletes the deprecated models such as BART. After this PR, Whisper will be the only encoder-decoder model that are supported by vLLM.