feat(mtp): accept llama.cpp dflash GGUFs as deepseek4 drafters, pull/validate treat drafter arches as companions

CHANGELOG.md

@@ -11,6 +11,8 @@ adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - Ctrl-T during a `chat` or `run` reply closes the model's open thinking
   block early (as if the thinking budget had just run out) so the answer
   starts now.
+- DeepSeek-V4 speculative decoding now accepts llama.cpp dflash draft GGUFs 
+  in addition to DwarfStar's.
 
 ### Changed
 

docs/arch-coverage.md

@@ -9,7 +9,7 @@ A GGUF loads iff gmlx maps its `general.architecture` to a `model_type` with a m
 | GGUF arch | model_type | model class | remap | synth | status | notes |
 |-----------|------------|:-----------:|-------|:-----:|--------|-------|
 | `deepseek2` | `deepseek_v3` | yes | yes | yes | loadable | DeepSeek-V3/R1 + GLM-4.x MLA conversions (GLM-4.7-Flash); MLA attention (absorbed embed_q/unembed_out via KQuantMultiLinear, native Q8_0), fine-grained sigmoid-gated MoE + shared expert + group routing, leading dense block. V2 (softmax gating) not yet supported |
-| `deepseek4` | `deepseek_v4` | yes | yes | yes | loadable | DeepSeek V4 Flash (256x8.4B, dwarfstar 'deepseek4' arch, not a llama.cpp conversion; parity reference is the ds4 engine). MLA-lite attention (low-rank q, single shared 512-dim KV latent K=V, grouped low-rank output proj, per-head fp32 sinks, tail-64 NEOX rope with inverse-rope'd output) in three per-layer variants from compress_ratios: sliding-window(128), +compressed pool, +lightning-indexer top-512 sparse. Manifold-constrained hyper-connections (4-stream 4D hidden, Sinkhorn mixing). Every-layer 256-expert MoE, sqrt-softplus gating + selection-only correction bias, first 3 layers hash-routed (tid2eid), shared expert, clamped SwiGLU. QAT round-trips (fp8-E4M3 KV, Hadamard+fp4-E2M1 indexer) reproduced on-path for logit parity. Model class vendored from mlx-lm PR #1192; separate MTP drafter GGUF (--draft-gguf) for speculative decoding |
+| `deepseek4` | `deepseek_v4` | yes | yes | yes | loadable | DeepSeek V4 Flash (256x8.4B, dwarfstar 'deepseek4' arch, not a llama.cpp conversion; parity reference is the ds4 engine). MLA-lite attention (low-rank q, single shared 512-dim KV latent K=V, grouped low-rank output proj, per-head fp32 sinks, tail-64 NEOX rope with inverse-rope'd output) in three per-layer variants from compress_ratios: sliding-window(128), +compressed pool, +lightning-indexer top-512 sparse. Manifold-constrained hyper-connections (4-stream 4D hidden, Sinkhorn mixing). Every-layer 256-expert MoE, sqrt-softplus gating + selection-only correction bias, first 3 layers hash-routed (tid2eid), shared expert, clamped SwiGLU. QAT round-trips (fp8-E4M3 KV, Hadamard+fp4-E2M1 indexer) reproduced on-path for logit parity. Model class vendored from mlx-lm PR #1192; separate MTP drafter GGUF (--draft-gguf, or autodetected next to the target) for speculative decoding - the DSpark sidecar in either container (gmlx deepseek4-dspark or llama.cpp dflash, e.g. the unsloth dspark release), or the legacy deepseek4_mtp_support nextn file |
 | `diffusion-gemma` | `diffusion_gemma` | yes | yes | yes | loadable | DiffusionGemma 26B-A4B; a non-autoregressive encoder-decoder block-diffusion model on the gemma-4 MoE backbone (each decoder layer runs a dense MLP + routed experts in parallel; encoder and decoder share weights bar a per-layer scalar). The decoder denoises a fixed-length canvas over reverse-diffusion steps with an entropy-bound sampler. Model class + denoising engine live in mlx-vlm (model_type diffusion_gemma); the backbone homes under model.decoder.* with the expert gate_up kept fused, plus the self-conditioning gated MLP. Single text-only GGUF (vision tower dropped at convert) |
 | `ernie4_5-moe` | `ernie4_5_moe` | yes | yes | yes | loadable | Baidu ERNIE-4.5-MoE (21B-A3B); fine-grained MoE (stacked routed experts -> SwitchGLU) + shared expert behind leading dense layers, softmax or aux-free sigmoid gating. NORM rope but mlx-lm uses traditional=True => Q/K pass through un-permuted (HF-native); the e_score_correction_bias is dropped on load (mlx-lm gates without it) |
 | `falcon-h1` | `falcon_h1` | yes | yes | yes | loadable | TII Falcon-H1 (0.5B-34B); parallel attention + Mamba2 in every layer (one input_layernorm feeds both, outputs summed), dense gated MLP under feed_forward.*, explicit head_dim from key_length. NEOX rope (no qk-permute). The muP multiplier zoo is folded into the wire weights at convert => synth pins every multiplier neutral; ffn_norm/ssm_a/ssm_d stored with no .weight suffix |

docs/cli.md

@@ -144,7 +144,7 @@ with a warning); `--no-speculative`/`--no-mtp` forces it off.
 |------|---------|
 | `--speculative` / `--mtp` | Force MTP speculative decoding on. Native-head models (qwen3.5/3.6 `nextn`) need no companion; gemma4 needs `--draft-gguf`. Native heads are auto-enabled without this. Use it to force the path when a sampler flag would otherwise defer. |
 | `--no-speculative` / `--no-mtp` | Disable MTP. Overrides the native-head auto-enable and config `speculative: true`. |
-| `--draft-gguf PATH` | Separate assistant-drafter GGUF (gemma4 two-GGUF MTP shape); implies `--speculative` (same as `serve`). |
+| `--draft-gguf PATH` | Separate assistant-drafter GGUF (gemma4 two-GGUF MTP shape, or a deepseek4 DSpark/MTP sidecar - gmlx `deepseek4-dspark`, llama.cpp `dflash`, or legacy `deepseek4_mtp_support`); implies `--speculative` (same as `serve`). A sidecar in the target's directory is autodetected without the flag. |
 | `--draft-block-size N` | Override the MTP draft block size. |
 
 Speculative generation takes only `--temp`/`--top-p`/`--top-k`/`--min-p` plus a

gmlx/arch_table.py

@@ -75,7 +75,7 @@ class ArchEntry:
     "mistral3":       ("llama",    "llama.cpp 'mistral3' = Ministral-3 / Mistral-Small-3.1, Llama layout"),
     "nemotron_h_moe": ("nemotron", "NVIDIA Nemotron-H MoE hybrid: Mamba2 SSM layers + sparse attention layers + MoE MLPs; layer_norm_epsilon (not rms_norm_eps)"),
     "deepseek2":       ("deepseek", "DeepSeek-V3/R1 + GLM-4.x MLA conversions (GLM-4.7-Flash); MLA attention (absorbed embed_q/unembed_out via KQuantMultiLinear, native Q8_0), fine-grained sigmoid-gated MoE + shared expert + group routing, leading dense block. V2 (softmax gating) not yet supported"),
-    "deepseek4":      ("deepseek", "DeepSeek V4 Flash (256x8.4B, dwarfstar 'deepseek4' arch, not a llama.cpp conversion; parity reference is the ds4 engine). MLA-lite attention (low-rank q, single shared 512-dim KV latent K=V, grouped low-rank output proj, per-head fp32 sinks, tail-64 NEOX rope with inverse-rope'd output) in three per-layer variants from compress_ratios: sliding-window(128), +compressed pool, +lightning-indexer top-512 sparse. Manifold-constrained hyper-connections (4-stream 4D hidden, Sinkhorn mixing). Every-layer 256-expert MoE, sqrt-softplus gating + selection-only correction bias, first 3 layers hash-routed (tid2eid), shared expert, clamped SwiGLU. QAT round-trips (fp8-E4M3 KV, Hadamard+fp4-E2M1 indexer) reproduced on-path for logit parity. Model class vendored from mlx-lm PR #1192; separate MTP drafter GGUF (--draft-gguf) for speculative decoding"),
+    "deepseek4":      ("deepseek", "DeepSeek V4 Flash (256x8.4B, dwarfstar 'deepseek4' arch, not a llama.cpp conversion; parity reference is the ds4 engine). MLA-lite attention (low-rank q, single shared 512-dim KV latent K=V, grouped low-rank output proj, per-head fp32 sinks, tail-64 NEOX rope with inverse-rope'd output) in three per-layer variants from compress_ratios: sliding-window(128), +compressed pool, +lightning-indexer top-512 sparse. Manifold-constrained hyper-connections (4-stream 4D hidden, Sinkhorn mixing). Every-layer 256-expert MoE, sqrt-softplus gating + selection-only correction bias, first 3 layers hash-routed (tid2eid), shared expert, clamped SwiGLU. QAT round-trips (fp8-E4M3 KV, Hadamard+fp4-E2M1 indexer) reproduced on-path for logit parity. Model class vendored from mlx-lm PR #1192; separate MTP drafter GGUF (--draft-gguf, or autodetected next to the target) for speculative decoding - the DSpark sidecar in either container (gmlx deepseek4-dspark or llama.cpp dflash, e.g. the unsloth dspark release), or the legacy deepseek4_mtp_support nextn file"),
     "glm-dsa":        ("deepseek", "GLM-5.2 (DeepSeek-V3.2): deepseek2 MLA + fine-grained sigmoid-gated MoE plus a per-layer DSA 'lightning indexer' (top-k sparse-attention key selection) + an MTP/nextn layer (dropped on load). Reuses the DEEPSEEK2 remap (indexer.* patterns appended) -> mlx-lm model_type glm_moe_dsa (subclasses deepseek_v32)"),
     "glm4moe":        ("glm",      "GLM-4.5 / 4.6 (incl. GLM-4.5-Air); standard MHA (qk-norm, partial rotary) + DeepSeek-V3-style fine-grained sigmoid-gated MoE (shared expert, group routing, leading dense block). NextN/MTP layer dropped on load"),
     "gpt-oss":        ("gpt-oss",  "OpenAI gpt-oss 20B/120B; MoE (no dense MLP) with per-head attention sinks, alternating sliding/full attention, YaRN rope, and native MXFP4 experts (packed-repacked in RAM, or zero-copy GGUF wire bytes for streaming/over-RAM - never dequantized). Attn/embed/output Q8_0; router/norms/biases F32"),

gmlx/discovery.py

@@ -44,7 +44,8 @@
 # hidden size it carries as a "backbone" field). The set is the fast path; the
 # backbone-field probe in `_looks_like_drafter` catches future naming.
 _DRAFTER_ARCHES = frozenset({"gemma4_assistant", "gemma4-assistant", "gemma4_mtp",
-                             "deepseek4-dspark"})
+                             "deepseek4-dspark",
+                             "dflash"})  # llama.cpp's arch name for DSpark
 _BACKBONE_FIELDS = ("backbone_embedding_length", "embedding_length_out",
                     "n_embd_backbone")
 
@@ -95,10 +96,16 @@ class ClassifiedGguf:
 
 
 # Classification
-def _looks_like_drafter(meta, arch: str | None) -> bool:
+def is_drafter_arch(arch: str | None) -> bool:
+    """Whether ``arch`` names a draft/assistant model - a speculative-decoding
+    companion for a target model, never a standalone servable model."""
     if arch in _DRAFTER_ARCHES:
         return True
-    if arch and ("assistant" in arch or "_mtp" in arch):
+    return bool(arch) and ("assistant" in arch or "_mtp" in arch)
+
+
+def _looks_like_drafter(meta, arch: str | None) -> bool:
+    if is_drafter_arch(arch):
         return True
     if arch:
         for suf in _BACKBONE_FIELDS:
@@ -305,7 +312,8 @@ def header_sampling(path) -> dict:
 
 def find_mtp_companion(
     path: str,
-    drafter_arch: str | tuple = ("deepseek4-dspark", "deepseek4_mtp_support"),
+    drafter_arch: str | tuple = ("deepseek4-dspark", "dflash",
+                                 "deepseek4_mtp_support"),
 ) -> str | None:
     """Path of an MTP drafter GGUF (arch in ``drafter_arch``) sitting in the
     same directory as ``path``, or ``None``. Header-only peeks through

gmlx/manage.py

@@ -165,7 +165,11 @@ def _verdict(ref: remote.Ref, report: remote.HeaderReport, *,
     # A LoRA adapter carries its base model's arch (general.type = "adapter"),
     # so like mmproj it's a valid companion file, never a standalone model.
     adapter = report.gguf_type == "adapter"
-    if mmproj or adapter:
+    # A draft model (MTP / DSpark / assistant) is likewise a companion: it
+    # rides a target model, so the standalone arch gate doesn't apply.
+    from .discovery import is_drafter_arch
+    drafter = not (mmproj or adapter) and is_drafter_arch(report.arch)
+    if mmproj or adapter or drafter:
         arch_ok, arch_err = False, None
     else:
         arch_ok, arch_err = _arch_status(report.arch, hf_source=hf_source)
@@ -180,12 +184,14 @@ def _verdict(ref: remote.Ref, report: remote.HeaderReport, *,
         # unknown); companions (mmproj/adapter) ride a base model, so their
         # standalone fit is not judged.
         "fit": (classify_fit(size_bytes, ram)
-                if size_bytes and not (mmproj or adapter) else None),
+                if size_bytes and not (mmproj or adapter or drafter)
+                else None),
         "arch": report.arch,
         "arch_supported": arch_ok,
         "arch_error": arch_err,
         "mmproj": mmproj,
         "adapter": adapter,
+        "drafter": drafter,
         "n_shards": n_shards,
         "n_tensors": report.n_tensors,
         "codecs": dict(sorted(report.histogram.items())),
@@ -195,7 +201,8 @@ def _verdict(ref: remote.Ref, report: remote.HeaderReport, *,
         # loadable = runs standalone; usable also admits a healthy companion
         # (mmproj / LoRA adapter).
         "usable": (report.loadable_codecs and arch_ok)
-                  or ((mmproj or adapter) and report.loadable_codecs),
+                  or ((mmproj or adapter or drafter)
+                      and report.loadable_codecs),
     }
 
 
@@ -207,6 +214,8 @@ def _print_report(v: dict) -> None:
         print(f"  architecture: {arch}  [mmproj companion]")
     elif v.get("adapter"):
         print(f"  architecture: {arch}  [LoRA adapter]")
+    elif v.get("drafter"):
+        print(f"  architecture: {arch}  [draft-model companion]")
     elif v["arch_supported"]:
         print(f"  architecture: {arch}  [supported]")
     else:
@@ -242,13 +251,18 @@ def _print_report(v: dict) -> None:
         print("  => adapter companion: a LoRA adapter, not a standalone model -")
         print("     attach it to its base GGUF: --adapter on run/chat/serve, or")
         print("     `adapter:` per model in the server config.")
+    elif v.get("drafter") and v["codecs_loadable"]:
+        print("  => draft-model companion: a speculative-decoding drafter, not a")
+        print("     standalone model - place it in the same directory as its")
+        print("     target GGUF (autodetected) or pass --draft-gguf.")
     else:
         reasons = []
         if not v["codecs_loadable"]:
             bad = ", ".join(f"{k}x{n}"
                             for k, n in v["unsupported_codecs"].items())
             reasons.append(f"unsupported codecs ({bad})")
-        if not v["arch_supported"] and not (v.get("mmproj") or v.get("adapter")):
+        if not v["arch_supported"] and not (v.get("mmproj") or v.get("adapter")
+                                            or v.get("drafter")):
             reasons.append("architecture not supported")
         print(f"  => not loadable: {'; '.join(reasons)}")
 
@@ -846,7 +860,8 @@ def cmd_pull(argv: list | None = None, prog: str = "gmlx pull") -> int:
             if not v["codecs_loadable"] and v["unsupported_codecs"]:
                 reasons.append(
                     "no kernel for codec(s): " + ", ".join(v["unsupported_codecs"]))
-            if not v["arch_supported"] and not (v.get("mmproj") or v.get("adapter")):
+            if not v["arch_supported"] and not (v.get("mmproj") or v.get("adapter")
+                                                or v.get("drafter")):
                 reasons.append(f"unsupported arch: {v['arch'] or '?'}")
             why = f" - {'; '.join(reasons)}" if reasons else ""
             print(f"\nrefusing to download an unloadable GGUF{why}. Pass --force to "

gmlx/mtp_load.py

@@ -477,6 +477,11 @@ def _load_deepseek4_mtp_drafter(
     arrays, kquant_meta, d_arch, _meta, _shapes = load_gguf_wire_bytes(
         draft_gguf_path, zero_copy=zero_copy
     )
+    if d_arch == "dflash":
+        arrays, kquant_meta, _meta = normalize_dflash_arrays(
+            arrays, kquant_meta, _meta
+        )
+        d_arch = "deepseek4-dspark"
     if d_arch == "deepseek4-dspark":
         return _load_deepseek4_dspark_drafter(
             draft_gguf_path,
@@ -489,9 +494,9 @@ def _load_deepseek4_mtp_drafter(
         )
     if d_arch != "deepseek4_mtp_support":
         raise ValueError(
-            f"{draft_gguf_path}: expected a deepseek4_mtp_support or "
-            f"deepseek4-dspark drafter GGUF for a deepseek_v4 target, got "
-            f"arch {d_arch!r}"
+            f"{draft_gguf_path}: expected a deepseek4-dspark, dflash, or "
+            f"deepseek4_mtp_support drafter GGUF for a deepseek_v4 target, "
+            f"got arch {d_arch!r}"
         )
     log(
         f"[mtp] drafter gguf ({d_arch}): {len(arrays)} arrays, "
@@ -586,6 +591,83 @@ def _load_deepseek4_mtp_drafter(
 }
 
 
+# llama.cpp packages the same DSpark drafter under arch "dflash" (its
+# convert_hf_to_gguf --dspark output, e.g. the unsloth release): per-stage
+# tensors under blk.{k}.* with the same leaf names, and drafter-level tensors
+# renamed through its root map. normalize_dflash_arrays translates that
+# container back to the deepseek4-dspark namespace so one remap serves both.
+# Drafter-level base name -> dspark base name; {L} is the last stage index
+# (stage placement only matters for the remap's per-stage bookkeeping:
+# main_proj/main_norm are stage-0 entries, the head tensors final-stage).
+_DFLASH_ROOT_MAP = {
+    "fc": "mtp.0.main_proj",
+    "enc.output_norm": "mtp.0.main_norm",
+    "output_norm": "mtp.{L}.norm",
+    "markov_w1": "mtp.{L}.markov_head.markov_w1",
+    "markov_w2": "mtp.{L}.markov_head.markov_w2",
+    "conf_proj": "mtp.{L}.confidence_head.proj",
+    "output_hc_fn": "mtp.{L}.hc_head_fn",
+    "output_hc_base": "mtp.{L}.hc_head_base",
+    "output_hc_scale": "mtp.{L}.hc_head_scale",
+}
+_DFLASH_SUFFIXES = (".weight", ".scales", ".biases", ".bias")
+
+
+def _dflash_rename(name: str, last_stage: int) -> str:
+    """The deepseek4-dspark name for one dflash tensor entry."""
+    if name.startswith("blk."):
+        return "mtp." + name[len("blk."):]
+    base, suffix = name, ""
+    for s in _DFLASH_SUFFIXES:
+        if name.endswith(s):
+            base, suffix = name[: -len(s)], s
+            break
+    mapped = _DFLASH_ROOT_MAP.get(base)
+    if mapped is None:
+        raise RuntimeError(
+            f"dflash normalize: unknown tensor {name!r} "
+            f"(the drafter tensor set is closed)"
+        )
+    return mapped.replace("{L}", str(last_stage)) + suffix
+
+
+def normalize_dflash_arrays(arrays: dict, kquant_meta: dict, meta: dict):
+    """Translate a llama.cpp ``dflash`` GGUF (tensor names and metadata) to
+    the ``deepseek4-dspark`` namespace. Returns ``(arrays, kquant_meta,
+    meta)`` ready for :func:`_load_deepseek4_dspark_drafter`."""
+    stages = {
+        int(n.split(".")[1]) for n in arrays if n.startswith("blk.")
+    }
+    if not stages:
+        raise RuntimeError("dflash normalize: no blk.* stage tensors found")
+    last_stage = max(stages)
+    n_arrays = {
+        _dflash_rename(name, last_stage): arr for name, arr in arrays.items()
+    }
+    n_kquant = {
+        _dflash_rename(name, last_stage): codec
+        for name, codec in kquant_meta.items()
+    }
+    n_meta = dict(meta)
+    block_size = meta.get("dflash.block_size")
+    if block_size is not None:
+        n_meta["dspark.block_size"] = block_size
+    # llama.cpp's converter writes the capture layers shifted +1 (its layer 0
+    # is the embedding) and carries the noise token as the tokenizer mask
+    # token; undo both. Verified against the source config ([40, 41, 42] for
+    # a 43-layer target) and conversion/deepseek.py's add_target_layers.
+    layers = meta.get("dflash.target_layers")
+    if layers is not None:
+        n_meta["dspark.target_layer_ids"] = [int(i) - 1 for i in layers]
+    mask = meta.get("tokenizer.ggml.mask_token_id")
+    if mask is not None:
+        n_meta["dspark.noise_token_id"] = int(mask)
+    w1 = n_arrays.get(f"mtp.{last_stage}.markov_head.markov_w1.weight")
+    if w1 is not None and "dspark.markov_rank" not in n_meta:
+        n_meta["dspark.markov_rank"] = int(min(w1.shape))
+    return n_arrays, n_kquant, n_meta
+
+
 def remap_deepseek4_dspark_arrays(
     arrays: dict,
     kquant_meta: dict,