Open
Description
🚀 The feature, motivation and pitch
Currently, when handling HTTP requests with multiple subrequests, the response only includes kv_transfer_params
for one subrequest, making it impossible to access KV transfer information for other subrequests.
Related PR: #17751
Reference code:
vllm/vllm/entrypoints/openai/serving_completion.py
Lines 514 to 520 in e384f2f
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.