Repository navigation
bloomery 0.2.5
bloomery 0.2.5: one fix. Under WSL2, Qwen3.8, GLM-5.3 and DeepSeek-V4.1 no longer stop a few tokens into a request.
Install
Linux x86-64 with AVX2 (glibc 2.34+), an NVIDIA GPU of compute capability 8.6+ and driver R580 or newer (CUDA 13).
On Windows, run inside WSL2.
curl -fsSL https://raw.githubusercontent.com/midagedev/bloomery/main/tools/release/install.sh | sh
bloomery-serve --hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL --port 8080Docker and Homebrew as in the README.
Fixed
- A request no longer hangs under WSL2 (#1). Adaptive residency's expert
swap waited on the copy stream for a word the host raised only between steps, and a driver that runs the copy and
the request on one hardware queue (WSL2 does) put the request's readback behind that wait. The staging window now
stays open whenever the host is not computing a layer, so the word is raised before the readback needs it. This
covers Qwen3.8, GLM-5.3 and DeepSeek-V4.1. The 0.2.4 workaround,BLOOMERY_RESIDENCY=off, is no longer needed.
Also in this release: the penalty window (repeat_penalty, presence_penalty, frequency_penalty) now sees the
prompt's last repeat_last_n ids, as llama-server's does, not only the generated ones.
Thanks to @avlp12 for the report and the logs.
Full changelog: v0.2.4...v0.2.5