Qwen3.8-27B Q4 on M6 Mac mini 24GB: context scaling, MTP and prompt-cache benchmarks

[lpalbou]

I’ve published a benchmark of Qwen3.8-27B Q4 on the M6 Mac mini 24GB, covering prefill, decoding with and without MTP, prompt-cache reuse, and memory use across context sizes.

Benchmark code and data: https://github.com/lpalbou/medium/tree/main/qwen-mtp-bench

The headline results are around 270 tok/s average prefill, 9.8 tok/s ordinary decoding, and up to 17.5 tok/s with MTP. Generation slows with increasing context, and the MTP benefit becomes smaller.

With the macOS GPU wired-memory limit raised to 20GB, I could use roughly 30K context. About 40K was possible in stress testing with a 21GB limit, with considerably less room left for the rest of the system.

What I’d particularly like to compare across machines is the balance between uncached prompt processing, cached continuation, and decoding. The M6’s prefill result is in the range of M4 Max measurements I’ve seen, but that does not make the two machines interchangeable: bandwidth, available memory and cache reuse change the comparison.

This also informs the broader architectural question behind the article: how much more useful could larger unified-memory configurations become if prefill compute continues to improve alongside memory bandwidth?

I’m looking for matched runs on an M5 Pro Mac mini, or other Apple Silicon configurations. Using the same benchmark settings would help distinguish hardware differences from differences in context length, quantization, cache state and software versions.

Please include those details with any comparison.

Full write-up and charts:
https://medium.com/@lpalbou/mac-mini-m6-qwen3-8-27b-a-glimpse-at-apples-local-ai-future-1b2d36aa2c16

Note: these measurements use MLX 0.32.2 and MLX-VLM 0.7.3; the exact configuration and reproduction commands are in the benchmark README. Comparisons with MLX-LM would also be useful.

[159753a52]

A rough cross-check while you wait for matched runs. llama.cpp's Apple Silicon thread (ggml-org/llama.cpp discussion #4167) has pp512 for Llama 2 7B Q4_0 on most chips. Scaling those by parameter count to a 27B-class model gives about:

chip (GPU cores)	Llama 2 7B pp512	scaled to ~27.8B
M4 Pro (20)	440 t/s	~107 t/s
M4 Max (40)	886 t/s	~215 t/s
M5 Pro (20)	1,621 t/s	~393 t/s
M5 Max (40)	3,220 t/s	~781 t/s

So your ~270 t/s on the M6 mini is a bit above a 40-core M4 Max, as you said, and an M5 Pro would be expected to read prompts roughly 1.5x faster than the M6 mini, if MLX tracks llama.cpp's Metal backend here. At those rates a 32K prompt is ~2 min on the M6 mini versus ~80 s on an M5 Pro, before the first token.

Caveats: different engine (llama.cpp Metal vs MLX), Q4_0 vs your Q4, pp512 vs your average over longer prompts, and Qwen3.8 27B keeps full attention in only 16 of its 64 layers, so long prompts slow down less than a dense model. I keep these per-chip numbers on a prompt-speed page, https://modelvram.com/llm-speed-calculator/ (I built it); a matched M5 Pro run would tell whether the scaling holds for MLX.