Feature request — surface --ctx-checkpoints / --swa-full / --slot-prompt-similarity flags for hybrid architectures (Qwen3-Next)

[bmaxim77]

Summary

LM Studio currently doesn't expose three flags that are critical for prefix-cache performance on hybrid-architecture models (Qwen3-Next in particular):

1. --ctx-checkpoints (default 32 — should bump to 128 for multi-session)
2. --checkpoint-every-n-tokens (default 8192 — 4096 is better for mid-prompt restore granularity)
3. --slot-prompt-similarity (not exposed at all)
4. --swa-full (not exposed at all)

Without these, users running Qwen3-Next via LM Studio's MLX path get no effective prefix cache (silent silent recompute on every turn — see separate mlx-lm issue). The GGUF path works if llama-server is invoked directly, but LM Studio's wrapper doesn't pass these flags.

Observed impact

On the same model (Qwen3-Next-80B-A3B-Instruct Q8_0, 128K context):

• Via LM Studio's default config: repeat of a 60K prompt → ~70-80 s TTFT (no cache)
• Via direct llama-server with flags above: repeat of a 60K prompt → ~1.0 s TTFT (54× faster)

Suggested UI changes

Load dialog / advanced settings

Add to "Advanced" section (or a new "Cache" tab):

• "Checkpoint reservoir size" (slider 16–256, default 128) → --ctx-checkpoints
• "Checkpoint granularity (tokens)" (dropdown 2048 / 4096 / 8192, default 4096) → --checkpoint-every-n-tokens
• "Prefix similarity threshold" (slider 0.1–0.9, default 0.3) → --slot-prompt-similarity
• "SWA full state" (checkbox, default checked for SWA models) → --swa-full

Auto-detect for hybrid architectures

Detect from GGUF metadata whether the model is hybrid (Qwen3-Next, future Mamba2/Jamba) and auto-enable --swa-full + bump --ctx-checkpoints to 128.

Warning for hybrid + MLX path

If user loads a Qwen3-Next MLX variant, surface an info banner:

"This model's hybrid architecture doesn't yet have working prefix cache on the MLX backend. For multi-turn workloads with cache benefit, use the GGUF variant (llama.cpp backend) instead."

Why this matters

Qwen3-Next is one of the strongest local-inference targets for agentic use (80B total / 3.9B active, fits on 128 GB consumer hardware). Its full potential on LM Studio is currently unreachable without manually running llama-server with the right flags. Bringing the UX under LM Studio would eliminate the "fall back to terminal" step many users face.

Measured configuration (reference)

Working llama-server invocation for 128 GB M4 Ultra:

llama-server \
  -m Qwen3-Next-80B-A3B-Instruct-Q8_0-00001-of-00002.gguf \
  --ctx-size 393216 --parallel 3 \
  --swa-full --ctx-checkpoints 128 \
  --checkpoint-every-n-tokens 4096 \
  --slot-prompt-similarity 0.3 \
  --n-gpu-layers 999

Wired RAM: 92.5 GB. Cache speedup: 40–200× depending on prompt size and overlap.

[FennyFatal]

I think I would argue that the Checkpoint reservoir size could go as low as 2-4 for things like gemma that will often generate a 4GB checkpoint, maybe even down to 0-1 for low RAM situations.

[clearbluejar]

🍿

[aedart]

The --swa-full seems to be very important for Gemma 4 models. Without it, it becomes very slow - it has to reprocess prompt(s) and that just wasteful.