server: fix context checkpoint restore for hybrid/recurrent models (DeltaNet/Mamba)

[Tongas]

Problem

Hybrid/recurrent models like Qwen3.5-27B and Qwen3.6-27B (Gated DeltaNet architecture) force full prompt re-processing on every conversation turn. A 15K-token conversation takes ~15 seconds per turn instead of milliseconds, making multi-turn agentic workflows (OpenCode, Claude Code) practically unusable.

This has been reported in #20225, #19394, #19690 with no fix yet.

Root cause

Two bugs in tools/server/server-context.cpp:

Bug 1 — checkpoint search uses cur.pos_min < pos_min_thold, but for recurrent models pos_min always equals the full sequence length, so this check always fails and no checkpoint is ever restored.

Bug 2 — checkpoint creation requires n_tokens >= 64, so short prompts never get a checkpoint created — even if Bug 1 were fixed, there would be nothing to restore.

Fix

Fix 1: For hybrid/recurrent models, use cur.pos_max <= pos_next instead of the SWA-based pos_min check when searching for a valid checkpoint.

Fix 2: Lower the checkpoint creation minimum from 64 to 4 tokens for hybrid/recurrent models so short prompts also get checkpointed.

Performance impact

Tested with Qwen3.6-27B Q4_K_M on RTX 3090, multi-turn via OpenCode:

	Before	After
Turn 2 prefill (12K context)	12,146 tokens re-processed (~11s)	31 new tokens only (115ms)
Turn 3+	Full re-process every turn	Incremental only

Multi-turn agentic workflows go from minutes of wait per turn to near-instant.

Implementation

Already implemented and tested in: spiritbuun/buun-llama-cpp#26

The patch is two small changes to tools/server/server-context.cpp:

--- a/tools/server/server-context.cpp
+++ b/tools/server/server-context.cpp
@@ -2598,6 +2598,12 @@
+                                            // for hybrid/recurrent models (DeltaNet, Mamba), pos_min always equals
+                                            // the full sequence length, so the SWA-based pos_min check always fails.
+                                            // use pos_max <= pos_next instead to find the most recent valid checkpoint.
+                                            if (llama_model_is_recurrent(model) || llama_model_is_hybrid(model)) {
+                                                return cur.pos_max <= pos_next;
+                                            }
                                             return cur.pos_min < pos_min_thold;

@@ -2842,1 +2845,3 @@
-                    do_checkpoint = do_checkpoint && (pos_min >= 0 && slot.prompt.n_tokens() >= 64);
+                    const int checkpoint_min_tokens = (llama_model_is_recurrent(model) || llama_model_is_hybrid(model)) ? 4 : 64;
+                    do_checkpoint = do_checkpoint && (pos_min >= 0 && slot.prompt.n_tokens() >= checkpoint_min_tokens);

— Gastón Parravicini

[Tongas]

@ggerganov This issue persists in build 8943 with Qwen3.6-27B.
The existing checkpoint logic in #16382 does not handle hybrid models correctly
because pos_min always returns the full sequence length for recurrent state.

I have a working fix tested on RTX 3090 with Qwen3.6-27B Q4_K_M via OpenCode:

• server: fix context checkpoint restore for hybrid/recurrent models spiritbuun/buun-llama-cpp#26

Turn 2 prefill goes from 12K tokens (~11s) to 31 tokens (115ms) with the fix.
Would you consider reviewing the patch?

[ggerganov]

The fix is incorrect. There is nothing to fix in llama-server. You just have to enable "preserve_thinking" for this model:

https://huggingface.co/Qwen/Qwen3.6-35B-A3B#preserve-thinking

[Tongas]

@ggerganov I tested with preserve_thinking=true but without
the checkpoint fix — the full re-processing still occurs:

Without fix (preserve_thinking=true):
Turn 1: checkpoints: 0 (none created, threshold 64 tokens not met)
Turn 2: "forcing full prompt re-processing" → 75 tokens processed

With checkpoint fix (preserve_thinking=true):
Turn 1: checkpoint created
Turn 2: "restored context checkpoint" → 51 tokens only

The root cause is two bugs in the checkpoint logic,
independent of thinking mode. Fix: spiritbuun/buun-llama-cpp#26