Running gpt-oss-20b for retain on 8 GB VRAM (Windows + llama.cpp) — any better small model?

[unsafePtr]

Sharing my setup for running gpt-oss-20b (currently #1 on the retain leaderboard at 81.2) on a modest GPU, and asking if there's a better local option.

Hardware

• RTX 5060, 8 GB VRAM, 32 GB DDR5
• Windows (llama.cpp CUDA build)

Retain (local llama-server)

Unsloth's F16 GGUF: unsloth/gpt-oss-20b-GGUF.

Full launch command:

llama-server.exe ^
  -m <models-dir>\gpt-oss-20b-F16.gguf ^
  --host 0.0.0.0 --port 11434 ^
  -ngl 999 --n-cpu-moe 18 -c 65536 ^
  -fa on --jinja ^
  --alias gpt-oss-20b-unsloth-f16 ^
  --cache-reuse 256 ^
  --chat-template-kwargs "{\"reasoning_effort\":\"low\"}" ^
  --temp 1.0 --top-p 1.0 --top-k 0 ^
  -t 12 --parallel 1

Why the key flags:

• -ngl 999 --n-cpu-moe 18 — offload everything to GPU, then spill 18 MoE experts back to CPU. Without CPU spill, MoE blows past 8 GB → 0xC0000409 OOM. With 18, ~1.2 GB VRAM headroom at 64k ctx.
• --cache-reuse 256 — Hindsight's retain prompt has a stable ~2.7k-token system+schema prefix. Enabling KV prefix reuse cut a dry-run-extract from 145s to 50s (~3×). Confirmed via cached_tokens=2737 on subsequent LLM calls in Hindsight logs.
• -fa on --jinja — flash attention + Jinja chat template rendering (needed for gpt-oss Harmony format).
• --chat-template-kwargs "{\"reasoning_effort\":\"low\"}" — pins reasoning effort at the server side; keeps output on the extraction path, not the thinking path.
• --alias gpt-oss-20b-unsloth-f16 — model name reported by /v1/models; matches my gateway's routing target.

Wrapped in a supervisor script that auto-restarts the server in 5s on crash.

Throughput: 33-35 tok/s. Consolidation calls (~26k in / 2.7k out): 90-125s.

Reflect (paid, via OpenRouter)

gpt-oss-120b won't fit locally, so reflect goes to openai/gpt-oss-120b via OpenRouter. Runs infrequently — ~$0.30/mo at my usage.

Question

Is there a smaller or denser model that scores close to gpt-oss-20b on retain and fits GPU-only (no CPU spill)? Anything in the Qwen3 / Phi-4 / distilled families worth testing against the Hindsight retain eval? Happy to run comparisons and report back.

[jeffreyflynt]

Worth a look: LiquidAI's Nanos family, specifically LFM2-1.2B-Extract are built for schema-constrained extraction, which is close to what your retain step is doing (stable prefix, structured output).

Might be a different angle on your constraint rather than just satisfying it. The whole reason you need --n-cpu-moe is that gpt-oss-20b's MoE layers get slow once they spill to CPU. Nanos are small/dense enough (1.2B) that they run fast on CPU alone, so the GPU-fit question may not come into play here.

https://huggingface.co/collections/LiquidAI/liquid-nanos

[unsafePtr]

There is now LFM2.5-8B-A1B. Are you using LiquidAI's models yourself? Was wondering also about gemma4-12b, which should theoretically fit 8gb VRAM with a bit of tunning and limiting context.
Just to highlight how my processing pipiline looks like, I have bifrost proxy in between, with fallback to API in case if local model is non-responsive.