Two community v3 artifacts published: nvfp4full+DFlash2 (v3 container) and an uncensored all-NVFP4 Swift build — both faster & smaller on one RTX 5090

[kaushikvira]

Hi all! 👋 Long-time NInfer user (production serving on a single RTX 5090, bare-metal Ubuntu). We published three v3 artifacts and want to share the numbers, since they show how far a single Blackwell card + NInfer goes.

---

1️⃣ kaushikvira/Qwen3.8-27B-swift15-nvfp4full-dflash2-NInfer-v3 ⭐ new — now the production profile

A Swift-1.5 build: ukisai/Swift-1.5-Qwen3.8-27b (RL+OPD post-training of Qwen3.8-27B — agentic/coding focus, penalized overthinking → fewer thinking tokens) → our all-NVFP4 quantization → z-lab DFlash2 + proposal head, native v3.

• 18.42 GiB, sha256 16f313c0…, weights only 18.0 GiB in VRAM, 1,590 objects
• Global-divisor normalization was exactly representable — 0.0000% re-encode error across all 272 fused groups (artifact 2️⃣ was ~2.7% worst-case)
• Same-session A/B vs 2️⃣ (fresh generations, temp 0): IFBench prompt-strict 69.0 (+2.7), GSM8K-200 95.0% (tie), decode 160.8 tok/s (+8.1%), prefill @ 200k 3,269 tok/s, device KV pool 308,736 tok, needle exact @ 250,031 tok (3 depths)
• Trade-off: not abliterated — Swift-1.5 carries base Qwen3.8 refusals (no community abliterated Swift-1.5 exists yet). For uncensored workloads, 2️⃣ remains the pick.

2️⃣ kaushikvira/Qwen3.8-27B-swift-abliterated-nvfp4full-dflash2-NInfer-v3

An uncensored build: d0xin's abliterated BF16 (Swift-Qwen3.8-27B-Uncensored-BF16, huihui-style refusal-direction removal — all credit to d0xin / ukisai / huihui-ai) → our own all-NVFP4 quantization → z-lab DFlash2 + proposal head, native v3.

• 18.42 GiB, sha256 74c97213…, weights only 18.0 GiB in VRAM
• Fused NVFP4 attention parents: llm-compressor gives each module its own weight_global_scale, but the native A4 input route requires one contiguous parent region per fused group (GDN qkvz = 16,384 rows, attn qkgv = 14,336). We wrote a small normalizer that unifies each group to a shared divisor with E4M3-RNE block-scale rescale (verified dequant-invariant to 2.7% worst-case) — then the v3 converter fuses the parents and the engine loads it unmodified. No fork, no C++ patch.
• Same-session A/B vs artifact 3️⃣ (fresh generations, temp 0): IFBench +1.3…+4.4 pp, GSM8K tie (95.5 vs 96.5, n=200 noise), decode 148.7 tok/s, prefill @ 200k 3,260 tok/s (+12%), device KV pool 308,736 tok
• Long-context verified: needle-in-haystack exact at 250,031 tokens at 10/50/90% depth
• Known quirk: at temperature 1.0 streaming tool-calls occasionally emit duplicates (all valid JSON; clean at temp 0)

3️⃣ kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3

The cometkim nvfp4full + DFlash2 artifact upgraded to the v3 container (bit-preserved weights, f76e19c0-era tooling). Production baseline since the v3 cutover.

• 20.55 GiB, sha256 74d2fc17…, DFlash2 (7 draft tokens) + proposal head embedded
• IFBench prompt-strict 65.0, GSM8K-200 96.5%, gate decode 146.4 tok/s, prefill @ 200k = 2,911 tok/s
• Device KV pool 278,912 tok (auto, k8v4) + 48 GiB host-KV arena → full 262k context at concurrency 4

---

Why this engine is remarkable

262k context on one 32 GB card, 160 tok/s speculative-decoded, hybrid GDN KV that barely grows with context, host-KV offload that just works — and the data-driven v3 loader means zero model-support patches. Thank you @Neroued 🙏

All three artifacts ship full credit chains, SHA256SUMS, and conversion contracts. Serving config: --kv-dtype k8v4 --spec dflash2 --draft-tokens 7 --lm-head-draft --host-kv-mib 49152 --max-concurrency 4 --vision.

Happy to share the quantization/normalization tooling if useful.