Skip to content

v0.28.0.dev20260903143203

@TheVerdman TheVerdman tagged this 03 Sep 14:25
While testing #644, I found that a max-length request could remain in
the waiting queue indefinitely on the non-paged path.
`single_sequence_estimate` reports exactly enough block-aligned cache
memory for the request, but vLLM's `BlockPool` reserves one physical
block as its `null_block`, leaving the scheduler one block short.

This adds one cache block to the reported capacity so the original
one-sequence capacity remains available after the null block is
reserved. Paged-attention reporting is unchanged. This complements #513,
which handled the same reservation during `max_model_len` auto-fit.

I added a focused regression test and ran the worker/platform tests plus
the full non-slow suite locally (`1848 passed, 18 skipped, 39
deselected`).

Signed-off-by: Andrew Verdiramo <220828289+TheVerdman@users.noreply.github.com>
Co-authored-by: Andrew Verdiramo <220828289+TheVerdman@users.noreply.github.com>
Assets 2
Loading