Name and Version
version: 10183 (3018a11)
built with GNU 15.3.0 for Linux x86_64
Operating systems
Linux
GGML backends
CUDA
Hardware
Any
Models
GLM-5.2 (GLM_DSA), possibly hy_v3/qwen35moe/step35 as well
Problem description & steps to reproduce
#25980 changed GLM_DSA (GLM-5.2) model loading so that NextN/MTP tensors (blk.78) are loaded whenever present in a GGUF, regardless of whether --spec-type draft-mtp is passed. Previously these tensors were simply never loaded unless speculative decoding was requested.
This is a backward-compatibility regression for existing bundled GGUFs: the same file, launched with the same command, now uses more VRAM/RAM than it did before upgrading llama.cpp — with no action taken by the user.
Why this matters
Users running large MoE quants (e.g. 744B-class GLM-5.2 models) close to their VRAM/RAM ceiling can go from "loads fine" to OOM / cudaMalloc failure purely from updating llama.cpp, with no config change on their end.
The only current mitigation is --no-mtp at conversion time, which for a model this size means a full re-download of source weights, re-quantization, and re-distribution for anyone sharing GGUFs publicly. That's a heavy cost compared to flipping a runtime flag.
There is no load-time way to skip the NextN/MTP block for an already-converted GGUF.
Origin
This was flagged by the PR author at merge time, in the "Regression" section of #25980:
Behavior change for existing bundled GLM-5.2 GGUFs, flagged deliberately: nextn tensors flip from never-loaded to loaded-when-present... This matches the merged hy_v3 behavior; if maintainers prefer a load-time gate instead, happy to discuss.
The same presence-probe pattern reportedly also applies to hy_v3 (#25395), qwen35moe, and step35, so this may not be GLM_DSA-specific.
Request
Could we get a load-time option to skip loading the NextN/MTP block for GGUFs that already contain it — e.g.:
only load blk.78 (or the arch-appropriate NextN block) when --spec-type draft-mtp is explicitly passed, or
a --no-mtp / --skip-mtp runtime CLI flag, similar in spirit to the existing conversion-time flag
so users aren't forced to reconvert/redistribute large models just to preserve their previous VRAM footprint.
Affected models: GLM-5.2 (GLM_DSA), possibly hy_v3/qwen35moe/step35 as well
Relevant PR: #25980
First Bad Commit
No response
Relevant log output
failed: out of memory
Name and Version
version: 10183 (3018a11)
built with GNU 15.3.0 for Linux x86_64
Operating systems
Linux
GGML backends
CUDA
Hardware
Any
Models
GLM-5.2 (GLM_DSA), possibly hy_v3/qwen35moe/step35 as well
Problem description & steps to reproduce
#25980 changed GLM_DSA (GLM-5.2) model loading so that NextN/MTP tensors (blk.78) are loaded whenever present in a GGUF, regardless of whether --spec-type draft-mtp is passed. Previously these tensors were simply never loaded unless speculative decoding was requested.
This is a backward-compatibility regression for existing bundled GGUFs: the same file, launched with the same command, now uses more VRAM/RAM than it did before upgrading llama.cpp — with no action taken by the user.
Why this matters
Users running large MoE quants (e.g. 744B-class GLM-5.2 models) close to their VRAM/RAM ceiling can go from "loads fine" to OOM / cudaMalloc failure purely from updating llama.cpp, with no config change on their end.
The only current mitigation is --no-mtp at conversion time, which for a model this size means a full re-download of source weights, re-quantization, and re-distribution for anyone sharing GGUFs publicly. That's a heavy cost compared to flipping a runtime flag.
There is no load-time way to skip the NextN/MTP block for an already-converted GGUF.
Origin
This was flagged by the PR author at merge time, in the "Regression" section of #25980:
Behavior change for existing bundled GLM-5.2 GGUFs, flagged deliberately: nextn tensors flip from never-loaded to loaded-when-present... This matches the merged hy_v3 behavior; if maintainers prefer a load-time gate instead, happy to discuss.
The same presence-probe pattern reportedly also applies to hy_v3 (#25395), qwen35moe, and step35, so this may not be GLM_DSA-specific.
Request
Could we get a load-time option to skip loading the NextN/MTP block for GGUFs that already contain it — e.g.:
only load blk.78 (or the arch-appropriate NextN block) when --spec-type draft-mtp is explicitly passed, or
a --no-mtp / --skip-mtp runtime CLI flag, similar in spirit to the existing conversion-time flag
so users aren't forced to reconvert/redistribute large models just to preserve their previous VRAM footprint.
Affected models: GLM-5.2 (GLM_DSA), possibly hy_v3/qwen35moe/step35 as well
Relevant PR: #25980
First Bad Commit
No response
Relevant log output
failed: out of memory