DSpark drafters on an 11 GB RTX 2080 Ti: Gemma 4 26B-A4B +76 % code / +24 % prose, and where DSpark loses

[homeofe]

Since DSpark landed (#25173), community drafters have appeared for several models. We measured three of them on consumer hardware with the whole quality and speed setup we use for our own local gateway, including the case where DSpark is slower than what the model already has.

Hardware: RTX 2080 Ti 11 GB (Turing, sm_75, PCIe 3.0 x16), Threadripper 3960X (24 cores), 4-channel DDR4-3200, 125 GiB, Linux.

How we measured: llama-server, 8 prompts (4 code tasks, 4 English prose), 256 tokens, temperature 0, one slot (-np 1). Confirmation runs: 3 interleaved rounds, 12 requests per category, median. A request counts only if other processes used under one core before and during it (the box is shared); contaminated passes were rerun. Quality: our hard panel (code with hidden tests, tool scenarios, German, long context, formats) and an agent-tools panel (multi-step tool use), card sampling, 3 seeds.

Gemma 4 26B-A4B UD-Q4_K_M with williamliao's DSpark drafter

Drafter: williamliao/dspark_gemma4_26b-a4b-it-GGUF, Q8_0 (1.3 GB). It loads as is: DFlash backbone with Markov head, block 7 with bonus anchor, its own 32000-token head with a d2t map.

llama-server -m gemma-4-26B-A4B-it-UD-Q4_K_M.gguf --jinja -c 32768 -np 1 \
  --spec-type draft-dspark -md gemma4_26b-a4b-DSpark-Q8_0.gguf -ngld 99 \
  --spec-draft-n-max 3 --spec-draft-p-min 0 --spec-draft-n-ubatch 32

Setting	Code tok/s	English tok/s	Acceptance code / English
no drafter	56.1	56.3
DSpark n-max 3 (3 rounds)	98.7 (+76 %)	69.7 (+24 %)	84.5 / 54.3 %
DSpark n-max 6 (1 round)	104.0 (+83 %)	56.6 (+0.4 %)	67.5 / 33.5 %
DSpark n-max 3 on the official b11719 release (1 round)	99.7	69.9	85.3 / 52.0 %

• Panels with DSpark n-max 3: hard 42 of 42 (no drafter: 42 of 42), agent-tools 35 of 36 (no drafter: 35 of 36, the same single miss).
• VRAM 9.9 GB with the drafter, so it fits an 11 GB card with 32K context.
• Greedy texts with the drafter differ from the no-drafter texts on 5 of 8 prompts, always at near-ties; the drafter verifies 4 tokens per step, which changes the batch shape of the target. The panels did not move.

Ling-3.0-flash IQ4_XS with inclusionAI's DSpark drafter

Drafter: inclusionAI/Ling-3.0-flash-dspark (safetensors), converted to GGUF Q8_0 with the stock convert_hf_to_gguf.py (no --dspark switch; that one is for DeepSeek-V4). Ling does not fit the card, so both rows use the same GPU expert cache from our fork (routed experts in RAM, hot experts in VRAM); only the drafter changes.

Drafter	Code tok/s	English tok/s	Acceptance code / English
built-in MTP, n-max 3	50.9	47.5	68 / 53 %
DSpark n-max 3, p-min 0 (3 rounds)	66.5 (+31 %)	51.9 (+9 %)	96 / 58 %
DSpark n-max 7, p-min 0.7 (code only)	78.3 (+54 %)	47.4 (0 %)	89 / 79 %

Panels with DSpark n-max 3 (context 8192): hard 39 of 42 (all three misses are the long-context task, which does not fit 8192), agent-tools 36 of 36.

Why DSpark helps so much here: Ling's MTP head loses accuracy with each extra step (per position 0.84 / 0.61 / 0.47 on English), which is what DSpark's Markov head is meant to fix.

Qwen3.6-35B-A3B UD-Q4_K_M with williamliao's DSpark drafter: slower on prose

Drafter: williamliao/Qwen3.6-35B-A3B-DSPARK-GGUF, Q8_0. Target with --n-cpu-moe 32. One round.

Drafter	Code tok/s	English tok/s
built-in MTP, n-max 2	88.6	77.2
DSpark n-max 3	99.8 (+12.7 %)	73.5 (-4.8 %)
DSpark n-max 7	109.9 (+24 %)	52.0 (-33 %)

Qwen3.6's own MTP head is already good, so DSpark only pays off on code here. We keep MTP 2 for general use.

What we take from it

1. Measure against what the model already has. Against no drafter (Gemma 26B) DSpark is a big win; against a strong MTP head (Qwen3.6) it is a trade between code and prose.
2. Block length 3 was the best general setting for all three; longer blocks win on code and lose on prose every time.
3. --spec-draft-p-min did not help us on prose with MTP either: acceptance rises, accepted length per step stays the same (details in Pushing MTP to the limit with --spec-draft-p-min (15% faster) #25198).
4. Run a quality panel when texts change, even with greedy decoding; here the panels stayed equal.

Raw numbers and scripts can be shared if useful. Thanks to williamliao and inclusionAI for publishing the drafters.

AI disclosure: the measurements were run by AI agents (Anthropic Claude, OpenAI Codex) under our direction on our hardware