0.1.44.1-poly.6

The eight added models answer several requests at once, and DeepSeek V4 Flash's MXFP4 experts run on the card without a converted copy in RAM.

What changed since 0.1.44.1-poly.5

• "parallel": N in strata-<model>.json, or --parallel N when you run setup, now works for GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V, up to 8 (issue 22). Before, strata-poly answered one request at a time and the server queued the rest. The server drives it as it drives the Strata project's engine: a request alone writes with guesses as before, and when a second comes the two write together in batch slots, each with its own position, recurrent state and attention rows, while the weights and the experts they share are read once for both. A long prompt being read gives way to the slots between its pieces, so a chat does not stall while someone else pastes a long document. On an RTX 5090 with Qwen3.6 UD-Q4_K_M, four 300-token chats at once came at 428 to 430 tokens/s in all, where one alone writes at 312 to 333. On an 8 GB laptop, where most experts run on the CPU, three Qwen3.6 UD-Q3_K_M chats at once came at 105 tokens/s together and at 79 one after the other. With the expert cache fixed and no guesses, each answer is the one it would get alone: strata-poly --check-batch compares them token by token, every token matched for all five architectures, and three chats through the server gave the same text at once as alone. With the default settings two runs of a chat already differ after a few dozen words, together or alone. How it works and how to check it: docs/BATCHING.md.
• DeepSeek V4 Flash's MXFP4 experts are multiplied on the card now. Before, the engine converted the four MXFP4 layers' down experts to Q5_0 at load, into RAM that cannot be paged back to the file: 6 GB with UD-IQ2_XXS, 1.38 GiB a layer with UD-IQ4_XS, which mattered on a PC whose RAM is smaller than the file. On the 5090 the engine's own RAM went from 5,959 MiB to 341 with UD-IQ2_XXS, a 24K-token prompt read at 1,091 tokens/s where 1,000 before, and writing speed is the same. On RTX 50 cards llama.cpp's FP4 tiles quantize the prompt's rows to FP4, as llama.cpp does, so greedy text can differ from 0.1.44.1-poly.5 after a few tokens.
• The README's speed chart has all eight added models, and there is an issue form for model requests.

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh, and START-HERE.bat --setup --parallel 3 turns on three requests at once.

Limits

• The eight added models are experimental.
• Requests in batch slots write without guesses, and --batch-groups and the other batching options of the Strata project's engine are not there yet.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.