[Bug]: Shape error encountered in speculative decoding when enable_lora=True
#4872
Labels
bug
Something isn't working
enable_lora=True
#4872
Your current environment
🐛 Describe the bug
I'd like to try out the recently added speculative decoding features. However, I'm encountering a shape error at the following line of code when my model has
enable_lora=True
, even if I'm not using a LoRA adapter in my request:vllm/vllm/spec_decode/batch_expansion.py
Lines 172 to 173 in c7f2cf2
Minimal reproduction with
enable_lora=True
(encounters a shape error):Minimal reproduction without
enable_lora=True
(runs without error):The text was updated successfully, but these errors were encountered: