Feature Description
Auto-detect MTP (Multi-Token Prediction) capability from GGUF metadata and tensor names for custom models.
Currently MTP support is only enabled for built-in models that have the "mtp" label hardcoded in server_models.json, or when users manually add --label mtp at pull time. Custom models (user.*) with MTP heads (e.g. Qwen3.5-4B-MTP-GGUF) should have this capability auto-detected at load time, matching how vision and tool-calling are already auto-detected from GGUF metadata.
The GGUF metadata contains the {arch}.nextn_predict_layers KV key and tensor names with the "nextn" prefix (e.g. blk.{bid}.nextn.eh_proj, blk.{bid}.nextn.enorm, nextn.pre_projection) - the canonical naming convention defined in llama.cpp's GGUF spec. The server should scan these and automatically apply the "mtp" label, so llamacpp_server.cpp passes --spec-type draft-mtp to llama-server.
Use Case / Motivation
Users who pull or register custom GGUF models with MTP support have to manually discover and pass --spec-type draft-mtp to llamacpp_server.cpp. This is not obvious - most users won't know that it needs to be explicitly enabled. Auto-detection from the GGUF file itself makes it transparent, just like vision and tool-calling.
Platform Relevance
All platforms
Additional Context
Extend read_gguf_capabilities() in gguf_capabilities.h to scan the {arch}.nextn_predict_layers KV metadata key and tensor names containing "nextn" (e.g. blk.{bid}.nextn.eh_proj, nextn.pre_projection) - the canonical naming convention used by llama.cpp's GGUF spec. After reading the KV metadata pairs, iterate over the tensor info entries and check each tensor name for the "nextn" prefix. If either the KV key or tensor name indicates MTP, set caps.mtp = true, which apply_gguf_capability_labels() then maps to the "mtp" label.
This reuses the existing call site in model_manager.cpp::populate_static_max_context_window() which already runs for all downloaded LLM models at load time.
Feature Description
Auto-detect MTP (Multi-Token Prediction) capability from GGUF metadata and tensor names for custom models.
Currently MTP support is only enabled for built-in models that have the
"mtp"label hardcoded inserver_models.json, or when users manually add--label mtpat pull time. Custom models (user.*) with MTP heads (e.g.Qwen3.5-4B-MTP-GGUF) should have this capability auto-detected at load time, matching howvisionandtool-callingare already auto-detected from GGUF metadata.The GGUF metadata contains the
{arch}.nextn_predict_layersKV key and tensor names with the"nextn"prefix (e.g.blk.{bid}.nextn.eh_proj,blk.{bid}.nextn.enorm,nextn.pre_projection) - the canonical naming convention defined in llama.cpp's GGUF spec. The server should scan these and automatically apply the"mtp"label, sollamacpp_server.cpppasses--spec-type draft-mtpto llama-server.Use Case / Motivation
Users who pull or register custom GGUF models with MTP support have to manually discover and pass
--spec-type draft-mtptollamacpp_server.cpp. This is not obvious - most users won't know that it needs to be explicitly enabled. Auto-detection from the GGUF file itself makes it transparent, just like vision and tool-calling.Platform Relevance
All platforms
Additional Context
Extend
read_gguf_capabilities()ingguf_capabilities.hto scan the{arch}.nextn_predict_layersKV metadata key and tensor names containing"nextn"(e.g.blk.{bid}.nextn.eh_proj,nextn.pre_projection) - the canonical naming convention used by llama.cpp's GGUF spec. After reading the KV metadata pairs, iterate over the tensor info entries and check each tensor name for the"nextn"prefix. If either the KV key or tensor name indicates MTP, setcaps.mtp = true, whichapply_gguf_capability_labels()then maps to the"mtp"label.This reuses the existing call site in
model_manager.cpp::populate_static_max_context_window()which already runs for all downloaded LLM models at load time.