🧠 Ornith-1.0-9B on a single 3090 — a Qwen3-Next dense hybrid that runs full 262K in 13.4 GB (🧪)

[noonghunna]

DeepReinforce dropped Ornith-1.0-9B, an agentic-coding RL fine-tune. We pulled the official Q4_K_M GGUF, served it on ik_llama.cpp (drafter-free ngram self-spec — the GGUF has no MTP head), and ran the full gate on one RTX 3090. It earns a slot for one reason — footprint — and surfaced two findings worth your time: it's a Qwen3-Next dense hybrid (not plain dense), and its weak cli-40 score is half a harness artifact.

Status: 🧪 experimental — slug ik-llama/ornith9b-single. Niche, not a default (see Takeaways).

📋 Results Card — 1× RTX 3090 (24 GB)

① Serving

Config	Spec-dec	KV / ctx	decode TPS (narr / code)	VRAM / card
ik-llama/ornith9b-single · Q4_K_M	ngram, drafter-free (ngram-map-k, ~0.59 acc)	q8_0 / 262K	102 / 103 (TTFT 144 ms)	13.4 GB
ref: gemma-4-12B gemma-12b-single-int8-mtp	MTP assistant n=4	bf16 / 256K	117 / 122	20.7 GB

verify-stress 8/8 (NIAH → 0.92×262K) · soak-continuous PASS (0 VRAM growth, 0/100 silent-empty, p50 104 TPS, 98% retention).

② Quality — core 8-pack → /150 (benchlocal-cli --full, temp 0.6)

pack (max)	Ornith-9B (off / on)	gemma-4-12B (off, ref)
toolcall-15	12 / 13	13
instructfollow-15	12 / 15	14
structoutput-15	14 / 14	14
dataextract-15	10 / 9	10
reasonmath-15	10 / 12	13
bugfind-15	14 / 9	14
hermesagent-20	12 / 11	11
cli-40	7 / 12 (13 hardened)	16
TOTAL	91 / 95	105

Single run per arm — treat ≤±5–7/150 as noise.

③ Takeaways

• It's a Qwen3-Next dense hybrid, not plain dense. arch qwen35 with full_attention_interval=4 → 8 full-attention + 24 GDN/DeltaNet layers (non-MoE). Only ¼ of layers carry attention KV, so the full 262K cache is just 4.25 GB — read the engine's KV self size; don't extrapolate per-layer × all-layers (overshoots ~4×).
• ngram spec-dec works despite DeltaNet. EAGLE/DFlash are KV-rollback-blocked on Qwen3-Next; ik_llama's drafter-free ngram-map-k accepts here (~0.59) and speeds up copy-heavy generation. No MTP head in the GGUF → ngram is the only path.
• The cli-40 score is half a harness artifact. Raw 7/40 looks damning for an "agentic-coding" model, but the transcripts show it solves the tasks (fixes the file, validates → ok) then mis-routes the <solution verdict="done"> terminator through the bash tool and loops to the turn cap. A hardened agent prompt ("emit the solution as a message, never via bash; stop once correct") recovers it 7 → 13/40. Real weakness (it needs the scaffold), not incapacity.
• Niche, not a default. gemma-4-12B beats it on quality (105 vs 91) and speed (117/122 vs 102/103) at +7 GB. Pick Ornith only when the lean 13.4 GB footprint / 16 GB-card fit is the deciding factor.

Requirements

• 1× 24 GB GPU (RTX 3090 / 4090 class); fits a 16 GB card at reduced context.
• ik_llama.cpp (the --spec-type self-spec build) — GGUF, so no transformers/vLLM.
• Sampling (per the card): temperature 0.6, top_p 0.95, top_k 20. (Their published benches used temp 1.0.)

Getting it / Run it

# official Q4_K_M GGUF: deepreinforce-ai/Ornith-1.0-9B-GGUF
MODEL_DIR=<your-models-dir> docker compose -f ngram.yml up -d   # ik-llama/ornith9b-single

# tune the ngram self-spec stage
NGRAM_NMAX=64 NGRAM_MIN_HITS=1 docker compose -f ngram.yml up -d

Credits

• Model + official GGUFs: deepreinforce-ai (Ornith-1.0-9B-GGUF)
• Engine + drafter-free ngram self-spec: ik_llama.cpp (ikawrakow)

---

What would help: cross-rig numbers (esp. the 16 GB-card context ceiling + ngram accept on your prompt mix), and a dual-card run of the 35B (qwen35moe, MoE) — that's the variant we think might actually contend with the 27B class.