Skip to content

v1.17.0

Choose a tag to compare

@github-actions github-actions released this 15 Aug 22:45
· 17 commits to main since this release

Added

  • Qwen3.8-27B (dense + MTP) in the shipped registry. Its NextN head is in the GGUF (qwen35.nextn_predict_layers=1, blk.64), so MTP speculative decoding works — measured draft acceptance ~0.87. Measured on an RTX 5070 Laptop (8GB) with UD-Q4_K_XL: 6.42 tok/s at 32k context, -ngl 18, 7042MB; 6.72 tok/s at 16k with -ngl 20. Being dense, it has no experts to park in RAM, so there is no --n-cpu-moe trick and it runs about 7× slower than Qwen3.6-35B-A3B on the same card — the quality option, not the fast one. One trap worth knowing: at 32k, -ngl 20 passes /health and then generates zero tokens. It fits in VRAM with nothing left to compute, so a server that started is not a config that works. Launch script: run/qwen38-dense.sh.
  • README sections for background jobs and per-phase model selection, both shipped in v1.16.0 without docs.