GLM-5.2 routed-Q4: decode fails after a long prefill with "Metal model range X..X GiB is not covered by mapped model views" (zero-length range)

[kuschpel]

GLM-5.2 routed-Q4: decode fails after a long prefill with "Metal model range X..X GiB is not covered by mapped model views" (zero-length range)

Summary

On Metal SSD streaming, GLM-5.2-UD-Q4_K_RoutedQ4K.gguf completes prefill of a ~64.7k-token prompt at ~33 t/s and then fails at the prefill→decode transition. The reported range has start == end, i.e. a zero-length lookup:

ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views

The request returns an empty response. A short prompt on the same model and the same build works correctly.

This is the same message class as #388 (closed 2026-06-16). This build is newer than that fix, so this looks like a remaining case in the GLM routed path rather than a regression of #388.

Environment

• ds4 commit efdadd41e20134af4f3381e1ed90e96fe4faef6f (2026-07-20), clean checkout, make default Metal build
• Apple M5 Max, 128 GiB unified memory, macOS 26.5.2 (25F84)
• iogpu.wired_limit_mb = 0 (system default)
• Model: antirez/GLM-5.2-GGUF → GLM-5.2-UD-Q4_K_RoutedQ4K.gguf, 434,170,886,208 bytes, size verified byte-exact against the Hub

Reproduction

Fails:

./ds4-server -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf \
  --ssd-streaming --ssd-streaming-cache-experts 48GB \
  --ctx 90112 --host 127.0.0.1 --port 8000
# then POST /v1/chat/completions with a single user message of ~64,700 tokens
# (German-language document set, "think": true, max_tokens 32768)

Server log, abridged:

ds4: GLM SSD streaming full resident layers: 0 (auto selected 0 layers)
ds4: metal SSD streaming total expert budget 48.00 GiB = 10.12 GiB prefill headroom + ... dynamic cache
...
ds4-server: chat ctx=0..64709:64709 prefill chunk  2048/64709 (3.2%)  avg=27.51 t/s
ds4-server: chat ctx=0..64709:64709 prefill chunk 59392/64709 (91.8%) avg=34.52 t/s
ds4-server: chat ctx=0..64709:64709 prefill chunk 64709/64709 (100.0%) avg=33.64 t/s   1923.816s
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views

Works (same model, same build, same binary):

./ds4 -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ssd-streaming \
  --ctx 4096 --nothink --tokens 20 -p "Antworte mit einem Satz: Was ist eine Arbeitsunfaehigkeit?"
# -> coherent German sentence, prefill 1.14 t/s, generation 1.73 t/s, no error

What was ruled out

Four configurations, identical failure at the identical point (prefill 100% → decode):

--ssd-streaming-cache-experts	--ctx	result
auto (capped at 12.00 GiB, 94 experts)	131072	prefill 42.1 t/s, then the error
64GB	90112	rejected at startup by the GLM memory guard (required 100.54 GiB, budget 96.00 GiB) — never reached decode
48GB	90112	prefill 33.6 t/s, then the error
32GB + DS4_GLM_STREAMING_DECODE_FULL_LAYER_MAP=1	90112	prefill 33.5 t/s, then the error

So the failure is independent of the expert cache budget and of context size, and forcing the full-layer decode map does not avoid it.

Notes that may help

• The failing range is zero-length (2.29..2.29 GiB). In ds4_metal.m the lookup requires offset >= view_start && end <= view_end; a zero-length range at an unmapped offset therefore falls through to the error path. Whether the real defect is the zero-length request itself or a missing view is not something I can tell from outside.
• 2.29 GiB is well past the initial streaming map, which the log reports as restricted to token embedding (1 spans, 0.94 GiB tensor span).
• The file is an Unsloth-dynamic ("UD") quant, i.e. per-layer non-uniform expert quantization — the same property --ssd-streaming: all requests fail on GGUFs with per-layer mixed routed-expert quantization ("Metal model range ... is not covered by mapped model views") #388 identified as the trigger ("Q4_K islands in an IQ2 sea").
• QA_BEFORE_RELEASES.md §7 regression-tests this exact message, but only for the mixed-quant Flash GGUF. A GLM routed-Q4 case does not appear to be covered.
• Prefill throughput on this machine is much higher than the README's "one Mac, SSD streaming" row suggests (~33–43 t/s measured vs ~3–5 t/s listed for GLM IQ2_XXS). If that row is stale, it may be worth refreshing.