Summary
#668 added live-KV rewind for repeated exact-prefix prompts (commit 7694112), but live_prefix_rewind_target is gated on ds4_engine_is_glm_dsa(), so DeepSeek V4 Flash on the Metal backend still does a full cold prefill when a client resends a prompt that is an exact token-level prefix of the live session.
Reproduction / evidence
Agent client (Hermes Agent, OpenAI chat/completions) aborts a stream mid-generation (user redirect). The slot's live KV then holds prompt + partial generation. The resent prompt is an exact prefix of that state — and is treated as a total miss:
ds4-server: live kv cache miss live=175286 prompt=175249 common=175249 reason=token-mismatch
ds4-server: chat ctx=0..175249:175249 prompt start
...
ds4-server: chat ctx=0..175249:175249 prompt done 643.560s
common == prompt_len (the whole 175,249-token prompt matches the live prefix; only the 37 partially-generated tokens diverge), yet the slot is discarded and ~10.7 minutes of prefill are redone. This happened 4 times in 15 minutes in a real session.
Why this seems generalizable
ds4_session_rewind() itself looks engine-agnostic (truncates the checkpoint, invalidates MTP/DSpark capture). The GLM gate appears to guard the dense-cache capping (ds4_session_glm_cap_dense_cache). Question: what needs validation to enable the rewind for the Flash/Metal session path — is there a Metal-graph or streamed-expert invariant that a truncated checkpoint violates?
Environment
Related: #609 / PR #611 (OpenAI tool-round continuation), #444 (disk KV eviction). Happy to test any branch — the repro environment is live here.
Summary
#668 added live-KV rewind for repeated exact-prefix prompts (commit 7694112), but
live_prefix_rewind_targetis gated onds4_engine_is_glm_dsa(), so DeepSeek V4 Flash on the Metal backend still does a full cold prefill when a client resends a prompt that is an exact token-level prefix of the live session.Reproduction / evidence
Agent client (Hermes Agent, OpenAI chat/completions) aborts a stream mid-generation (user redirect). The slot's live KV then holds
prompt + partial generation. The resent prompt is an exact prefix of that state — and is treated as a total miss:common == prompt_len(the whole 175,249-token prompt matches the live prefix; only the 37 partially-generated tokens diverge), yet the slot is discarded and ~10.7 minutes of prefill are redone. This happened 4 times in 15 minutes in a real session.Why this seems generalizable
ds4_session_rewind()itself looks engine-agnostic (truncates the checkpoint, invalidates MTP/DSpark capture). The GLM gate appears to guard the dense-cache capping (ds4_session_glm_cap_dense_cache). Question: what needs validation to enable the rewind for the Flash/Metal session path — is there a Metal-graph or streamed-expert invariant that a truncated checkpoint violates?Environment
ds4-server -m ds4flash.gguf -c 262144 --host 127.0.0.1 --port 8001 --kv-disk-dir /tmp/ds4-kv --kv-disk-space-mb 8192 --batched-session 4 --mtp .../DeepSeek-V4-Flash-DSpark-support.gguf --dsparkRelated: #609 / PR #611 (OpenAI tool-round continuation), #444 (disk KV eviction). Happy to test any branch — the repro environment is live here.