Skip to content

I use VLLM to deploy DeepSeek, and when there are high concurrency requests, many of them are pending reqs. How can I set a timeout for these waiting requests? #16392

nvliajia announced in General
I use VLLM to deploy DeepSeek, and when there are high concurrency requests, many of them are pending reqs. How can I set a timeout for these waiting requests? #16392
@nvliajia nvliajia
Apr 10, 2025 · 0 comments
Return to top

nvliajia
Apr 10, 2025

You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant
@nvliajia