fix(mp): correct store cached requests in lmcache_mp_connector

[maobaolong]

Summary

This PR fixes two bookkeeping bugs in lmcache_mp_connector_0180.py that together cause KV cache blocks to be silently dropped during store operations.

Here is a relevant vllm side PR vllm-project/vllm#39655 , the changed content is totally same with this PR.

---

Bug 1: GetStoreMetadata — incorrect min_available_blocks upper bound

Root cause (causes missed stores):

In GetStoreMetadata, min_available_blocks was computed solely from num_scheduled_tokens // vllm_block_size. However, num_stored_blocks (set during the lookup phase) already includes the retrieve-hit blocks. When a request has LMCache hits, the subtraction min_available_blocks - num_stored_blocks can go negative, preventing any new blocks from being stored.

Example (block_size=64, chunk_size=256, blocks_in_chunk=4):

Suppose a request has 4096 tokens, and LMCache lookup hits 60 blocks (3840 tokens). The scheduler then schedules the remaining 256 tokens (= 4 blocks).

Variable	Old code	New code
num_scheduled_tokens // block_size	256 / 64 = 4	4
num_lmcache_hit_blocks	—	60
computed_blocks	4	4 + 60 = 64
min_available_blocks	4	64
num_stored_blocks (from lookup)	60	60
num_staging_blocks	4 − 60 = −56 ❌	64 − 60 = 4 ✅

With the old code, num_staging_blocks is negative, so no new block is ever stored. The fix adds num_lmcache_hit_blocks to computed_blocks so the upper bound matches the baseline that num_stored_blocks was set against. Hit blocks are not re-stored because they are already counted in num_stored_blocks.

---

Bug 2: _process_cached_requests — cumulative vs. incremental token count

Root cause (bookkeeping corruption):

In _process_cached_requests, num_new_tokens was read from cached_reqs.num_computed_tokens[idx], which is a cumulative value (total computed tokens so far). It was then passed to increase_num_scheduled_tokens(), which does += (accumulation). This double-counts tokens across scheduling rounds.

Example (block_size=64, chunk_size=256):

Suppose a request has 4096 tokens and the scheduler splits it into two prefill rounds of 2048 tokens each:

Round	Source	Value passed	Tracker after +=
Round 1 (new_req)	num_scheduled_tokens	2048	2048 ✅
Round 2 (cached_req, old)	num_computed_tokens (cumulative)	4096	2048 + 4096 = 6144 ❌
Round 2 (cached_req, new)	num_scheduled_tokens (incremental)	2048	2048 + 2048 = 4096 ✅

The inflated tracker value (6144 instead of 4096) does not cause out-of-bound stores thanks to the min() guard in GetStoreMetadata, but it corrupts internal bookkeeping and is a latent bug. The fix switches to scheduler_output.num_scheduled_tokens[request_id] (incremental), consistent with _process_new_requests.

---

Changes

• GetStoreMetadata: include num_lmcache_hit_blocks in computed_blocks to restore the correct upper bound for num_staging_blocks.
• _process_cached_requests: use scheduler_output.num_scheduled_tokens[request_id] (incremental) instead of cached_reqs.num_computed_tokens[idx] (cumulative).

---

Note

Medium Risk
Changes KV-cache store bookkeeping for cached and partially-hit requests; mistakes here could still cause missed stores or incorrect block accounting during prefill/generation.

Overview
Fixes LMCache MP store bookkeeping so KV blocks aren’t silently skipped when requests have a mix of vLLM APC hits and LMCache hits.

GetStoreMetadata now computes the storeable-block upper bound using scheduled blocks plus max(num_vllm_hit_blocks, num_lmcache_hit_blocks) (with added rationale comments), and _process_cached_requests now increments num_scheduled_tokens using per-step scheduler_output.num_scheduled_tokens[request_id] instead of a cumulative computed-token counter.

^{Reviewed by Cursor Bugbot for commit a8ee866. Bugbot is set up for automated code reviews on this repo. Configure here.}

[gemini-code-assist]

Code Review

This pull request modifies the block calculation logic in GetStoreMetadata to include hit blocks and updates _process_cached_requests to use incremental scheduled tokens for consistency. A review comment was provided requesting the addition of regression tests for these bug fixes, as mandated by the repository style guide.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit d1e4179. Configure here.}

[ApostaC]

@maobaolong good catch for the bugs! Can you also create a PR in vLLM? Both me and @KuntaiDu can approve that for you!

[ApostaC]

LGTM!

[maobaolong]

@ApostaC Thanks for this quick review!

[sammshen]

LGTM!

[ApostaC]

@maobaolong Are we going to keep track with vllm-project/vllm#39719?

[maobaolong]

@maobaolong Are we going to keep track with vllm-project/vllm#39719?

Yeah, I will.