While benchmarking our GLM fork line against main we found that the whole
glm5.2 branch decodes DeepSeek Flash about 2.8x slower than main, and bisected
it to the first commit of the line.
Machine: Apple M5 Pro 64 GB, macOS 26 (Darwin 25.5.0), Metal.
Model: DeepSeek-V4-Flash IQ2XXS (w2Q2K, AProj/SExp/Out Q8, chat-v2 imatrix),
80.8 GiB file, so model >> RAM and --ssd-streaming. Probe:
./ds4-bench -m MODEL.gguf --ssd-streaming \
--prompt-file speed-bench/promessi_sposi.txt \
--ctx-start 2048 --ctx-max 2048 --step-incr 2048 --gen-tokens 32
Single-commit points are ctx 2048 only, rebuilt via make after each
checkout, and their ranges combine two measurement sessions on separate
days; the "main" and "our line" rows are interleaved arms from one session
(A/B/C x2 back to back) swept at ctx 2048-4096.
| commit |
gen t/s |
| main (5b95fa1) |
6.8-8.3 (4 interleaved runs, ctx 2048-4096) |
| main (80ebbc3, current HEAD) |
6.49 (1 run) |
| 173a9a3 "Add GLM 5.2 runtime support" |
1.94-2.21 |
| 34b3736 (pre pro-correctness merge) |
2.76-2.88 |
| bd89932 (glm5.2 HEAD) |
2.73 (1 run) |
| our GLM branch (bd89932 + our fixes, default env) |
1.9-2.8 (8 interleaved runs, ctx 2048-4096) |
A clean detail for bisecting: 80ebbc3 (current main, fast) is the direct
parent of 173a9a3 (slow), so the whole regression sits inside that one
commit. It persists through the branch HEAD, is untouched by streaming env
configuration (disabling the expert readahead does not move it), and main
is not affected. First token on the glm5.2 line takes ~5.2-7.2 s
(gen_first_ms 5166-7180 across all runs); main's bench build predates the
first-token/steady columns, so for main only whole-generation t/s is
directly comparable, and on the glm5.2 line even the steady phase after
the first token (gen_steady_tps 2.6-6.5) stays below main's whole-run
6.8-8.3.
Prefill shows no clear regression once the page cache is warm (fork line
107-120 vs main 116-118 t/s at these sizes in the warm interleaved reps;
first-rep deltas are cache-warming noise).
We have not root-caused inside 173a9a3; happy to bisect deeper into it or
test patches on this box. Repro scripts available.
While benchmarking our GLM fork line against main we found that the whole
glm5.2 branch decodes DeepSeek Flash about 2.8x slower than main, and bisected
it to the first commit of the line.
Machine: Apple M5 Pro 64 GB, macOS 26 (Darwin 25.5.0), Metal.
Model: DeepSeek-V4-Flash IQ2XXS (w2Q2K, AProj/SExp/Out Q8, chat-v2 imatrix),
80.8 GiB file, so model >> RAM and --ssd-streaming. Probe:
Single-commit points are ctx 2048 only, rebuilt via make after each
checkout, and their ranges combine two measurement sessions on separate
days; the "main" and "our line" rows are interleaved arms from one session
(A/B/C x2 back to back) swept at ctx 2048-4096.
A clean detail for bisecting: 80ebbc3 (current main, fast) is the direct
parent of 173a9a3 (slow), so the whole regression sits inside that one
commit. It persists through the branch HEAD, is untouched by streaming env
configuration (disabling the expert readahead does not move it), and main
is not affected. First token on the glm5.2 line takes ~5.2-7.2 s
(gen_first_ms 5166-7180 across all runs); main's bench build predates the
first-token/steady columns, so for main only whole-generation t/s is
directly comparable, and on the glm5.2 line even the steady phase after
the first token (gen_steady_tps 2.6-6.5) stays below main's whole-run
6.8-8.3.
Prefill shows no clear regression once the page cache is warm (fork line
107-120 vs main 116-118 t/s at these sizes in the warm interleaved reps;
first-rep deltas are cache-warming noise).
We have not root-caused inside 173a9a3; happy to bisect deeper into it or
test patches on this box. Repro scripts available.