v.0.5.9-alpha

Install command, run in PowerShell. From an admin PowerShell it installs for the whole machine instead.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux, with sudo sh at the end to install for the whole machine:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

• Mixture-of-experts models are faster with many requests at once. Tokens per second of all streams together, steady decode, on one RTX PRO 6000 Blackwell, every provider on the same NVFP4 checkpoint, thinking on, 8,192-token budget (the README's Batching section has the method and the charts):

streams at once	1	2	4	8	16	32
Qwen3.6 35B-A3B, llmash 0.5.9	510	774	1,214	1,936	2,985	4,103
Qwen3.6 35B-A3B, llmash 0.5.8	521	756	1,062	1,775	2,764	3,863
Qwen3.6 35B-A3B, vLLM 0.32	377	664	1,161	1,910	2,616	3,511

llmash is now ahead of vLLM at every level, by 35, 17, 5, 1, 14 and 17%; at 8 streams that is inside the 3% that repeated waves differ by. The single stream's lower figure is acceptance noise: its rounds are 3% faster than 0.5.8's. Gemma 4 E4B, a dense model, is unchanged.

• A request that names no temperature gets the model's own: the one its GGUF carries, as before, else the Ollama store's params, else 1.0, the value vLLM falls back to. It was llama.cpp's 0.8.

Runtime

The runtime changed, build 777 (fork commit 5070592c8), so llmash update replaces the one you have.

• A shared expert shaped like the routed experts (Qwen3.5 and 3.6 MoE) is loaded as one more slot of the expert tensors, in the same memory, and runs in the expert kernels with its gate as the weight, where it was a separate feed-forward block.
• An MoE's gate and up run as one launch over one sort of the expert ids and one quantization of the activations; the per-expert scales and the SwiGLU take one pass, the down projection reuses the sort, and its scales are read by expert id in the weighted sum.
• Small weights over a few sequences' rows (a router, the delta-net gating projections) split their K across the SMs and sum the parts in a fixed order, where a few SMs walked all of K.

Checked before release:

• The client program and its tests on Windows and Linux.
• 16 different requests at once, each asked to repeat a sentence only it was given, three rounds, on Qwen3.6 35B-A3B and Gemma 4 E4B: every reply holds its own sentence and no other.
• test-backend-ops for matrix multiplication, expert matrix multiplication, concatenation and fill.
• Wikitext-2 perplexity of Qwen3.6 35B-A3B: 5.8791, and 5.8751 on the same build with the shared expert computed apart.
• Not checked: the same greedy text as the previous runtime. The shared expert goes through the expert kernels' quantization and the small products sum in another order, so the numbers move slightly.