🔓 Deckard-40B (uncensored) on dual 3090 — results + a lossless-MTP finding (✅)

[noonghunna]

Updated 2026-06-10: promoted 🧪 → ✅ Production. The one open blocker — the model's architecture — is now confirmed from the GGUF header (qwen35-dense, standard GQA, 97 layers), and the full validation gate passed (verify-stress added). Cross-rig numbers are still very welcome — see "What'd help".

---

We just shipped llamacpp/deckard40B-dual-mtp — the catalog's first uncensored model, and our first dual-GPU llama.cpp compose. It's Qwen3.6-40B-Deckard — @DavidAU's dense 40B Opus-Deckard merge (original GGUF) — Q6_K, with a BF16 MTP head injected by @PiehSoft, layer-split across two 3090s. All credit for the model itself goes to DavidAU; PiehSoft made the self-speculative MTP path possible — see Credits.

The headline: mainline llama.cpp's newly-merged --spec-type draft-mtp is a ~2× decode on this dense model — and it's lossless.

🎴 Results Card — 2× RTX 3090, mainline llama.cpp b9570, think-off

① Serving

Config	Spec-dec	KV	ctx	Narr	Code	Accept	VRAM
baseline	none	fp16	16K	22.7	22.7	—	34 GB
MTP n=2 ⭐	draft-mtp	q8_0	128K	36	46	76–80%	42 GB @128k

n-sweep optimum n=2 (n2 41.6 ≈ n3 41.4 > n4 39.8 > n5 37.8 agg) · context ceiling 128K (q8_0 KV; 192K OOMs) · MTP = +59% prose / +104% code.

② Quality — core 8-pack (/150, think-off) — full breakdown

Pack	MTP-off	MTP-on
toolcall-15	15	14
instructfollow-15	14	15
structoutput-15	13	13
dataextract-15	10	10
reasonmath-15	10	11
bugfind-15	13	12
hermesagent-20	15	15
cli-40	15	15
Total /150	105 (70%)	105 (70%)
det /75 · sandbox /75	62 · 43	63 · 42

Exactly 105 = 105 — draft-mtp is distribution-preserving: the per-pack deltas (toolcall −1, instructfollow +1, reasonmath +1, bugfind −1) are single-case sampling jitter that nets to zero. So MTP buys the 2× speed at no quality cost. On par with Qwen3.6-27B (105–112 band). Also: verify-full 8/8 · verify-stress 8/8 (ceiling ladder filled to 120K / 91% of n_ctx, needle recall clean through 90K, 0 MiB VRAM growth) · soak-continuous PASS (0 VRAM growth, 0/25 silent-empty, 25 turns).

③ Takeaways

• Dense + MTP = the big-win case (no MoE-routing penalty on the draft head) → ship it with MTP on.
• A 40B doesn't out-score the 27B on the 8-pack — the draw here is uncensored + the MTP speed, not raw capability. reasonmath/dataextract reflect the think-off tax on a reasoning model.
• Dual-only (31 GB Q6 > 24 GB → both cards, layer-split, no NVLink needed); on a 2-GPU box it's mutually exclusive with single-card work.

Why an uncensored model?

For authorized security and research work, refusal friction is a real cost. An uncensored local model is a better fit when you're:

• Auditing / red-teaming — analyzing malware, reasoning about exploit chains and PoCs in a sanctioned pentest, reviewing offensive tooling — where a safety-tuned model hedges or declines even with clear authorization.
• Vulnerability research & CTF — deobfuscation, crypto/attack-path reasoning, without the loop breaking on flaggable-but-legitimate prompts.
• Defensive content research — studying harmful content to build defenses against it, where the refusal is the obstacle.
• Working on confidential targets — it runs entirely on your own GPUs: audit data, client material, and prompts never leave the box. For security work, that "nothing leaves the rig" property is often the bigger win than the uncensoring itself.

✅ Why it's Production

The original post shipped this 🧪 Experimental on two reservations — unconfirmed architecture fields and dual-only. Both are resolved:

• Architecture confirmed. The GGUF header reports general.architecture=qwen35 — standard GQA, 97 layers, uniform attention (not a Qwen3-Next/DeltaNet hybrid, as an early scaffold had guessed). The catalog profile now reflects qwen35-dense.
• Dual-only is a topology fact, not a defect. The 31 GB Q6_K can't fit one 24 GB card, so it layer-splits across both — same as plenty of our ✅ dual composes. On a 2-GPU box it's mutually exclusive with single-card / image-studio work; that's the trade, not a flaw.

Every gate is green on our rig (verify-full 8/8, verify-stress 8/8, 8-pack 105/150 lossless, soak-continuous PASS), so it ships ✅ Production. Cross-rig validation isn't a gate for ✅ here, but other-rig numbers are still genuinely useful — see "What'd help".

Getting it

The MTP-injected GGUF is public: PiehSoft/Qwen3.6-40B-Deckard-MTP-Q6_K — PiehSoft's BF16 MTP (nextn) head injected onto DavidAU's Opus-Deckard Q6_K base. It's in the catalog as llamacpp/deckard40B-dual-mtp (✅) and setup.sh/launch.sh auto-fetch it, or grab it directly with hf download PiehSoft/Qwen3.6-40B-Deckard-MTP-Q6_K. You'll want mainline llama.cpp b9570+ for --spec-type draft-mtp.

Run it

# (grab the GGUF first — see "Getting it" above; setup.sh/launch.sh auto-fetch it)
bash scripts/switch.sh --owui llamacpp/deckard40B-dual-mtp   # launches on :8199 AND auto-wires Open WebUI

It serves an OpenAI-compatible API on :8199 (model deckard-40b), using both 24 GB cards. Point any OpenAI client at it — aider, opencode, Continue, a raw curl http://localhost:8199/v1/models.

Tip — gpu-mode users: if you drive the stack with the gpu-mode switcher, there's now a one-word mode:

gpu-mode deckard      # or: bash scripts/gpu-mode.sh deckard

It frees both GPUs (stops any other dual-card model), boots Deckard on :8199, brings up the support services, and wires Open WebUI — in one shot. Switching to any other GPU mode (gpu-mode 27b, gemma, off, …) tears Deckard back down so the cards are free.

For Open WebUI, the --owui flag above does it for you: when OWUI is running, switch.sh registers the endpoint as a connection automatically (it's a no-op if OWUI isn't up — no harm), and deckard-40b shows up in the model picker. (If your OWUI runs on a different host, add it by hand instead: Admin → Settings → Connections → http://<rig-host>:8199/v1, key any non-empty string.) Verified: OWUI lists deckard-40b and chats round-trip to it.

What'd help

Cross-rig numbers (other 3090 pairs / 4090s — fit, TPS, MTP acceptance), uncensored-behaviour notes, and long-context past 128K (q4_0 KV). The genuinely portable finding: the lossless draft-mtp ~2× holds on any Qwen3.6-family dense GGUF, not just Deckard — worth a try if you run one.

Credits

• The model: @DavidAU — the Qwen3.6-40B Opus-Deckard uncensored frankenmerge and the Q6_K iMatrix GGUF we build on (Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NEO-CODE-Di-IMatrix-MAX-GGUF). All of the model's capability and character is DavidAU's work.
• The MTP head: @PiehSoft — injected the BF16 MTP (nextn) predict-layer onto DavidAU's Q6_K, which is what unlocks the lossless --spec-type draft-mtp ~2× decode reported above.
• The engine: mainline llama.cpp — --spec-type draft-mtp (PR #22673).

[noonghunna]

🔧 Tuning update — anti-loop defaults (shipped)

A user (milano, Discord) hit repetition loops at long context with deckard-40b. Root-caused + fixed:

• The compose shipped --repeat-penalty 1.0 (no penalty) + llama.cpp's default repeat-last-n 64 → degenerate loops (a phrase repeating up to ~46× in a 2500-token gen, intermittent at temp 0.6).
• New defaults: --repeat-penalty 1.1 + --repeat-last-n 256 → cuts looping ~5× (max 46 → ~9 in a per-request sweep), gentle on code.
• DRY sampler wired as opt-in (DRY_MULTIPLIER=0.8, default off) for severe long-ctx loops — strongest loop-breaker but over-suppresses legitimate repetition in code, so not default-on.
• Quality-neutral: same-session symmetric 8-pack A/B was 99 (old rep1.0) vs 100 (new rep1.1) — +1, within noise. No capability cost.

Note: this compose ships --reasoning off by default, so --reasoning-budget (a common anti-loop suggestion) is a no-op here unless you enable reasoning.

git pull to get it.