PolyStrata 0.1.42-poly.3

Two cards, Tiel Coder 35B-A3B, CUDA 12, and faster answers on the smaller cards.

What changed since 0.1.42-poly.2

• Two NVIDIA cards split a model's layers (--gpus 0,1 at setup, --layer-split on the engine). On two RTX 3080 (10 GB each) Qwen3.6 UD-Q3_K_M keeps every expert in VRAM and writes 215-221 tokens/s where one card writes 157-177, and reads a prompt at 3,100-3,560 where 1,570-1,700; Gemma 4 writes 189-193 where 178-187. GLM 5.3 Flash REAP50 runs on the pair too. Three cards and Windows with two cards were not tried. Details and the exactness checks: docs/MULTI_GPU.md.
• Tiel Coder 35B-A3B is installed by setup (--family tiel): Ornith 1.5 quantized for coding by peculiar-ragdoll, three sizes, with its draft block. u/Sensitive_Song4219 found that it runs on PolyStrata. On the 8 GB laptop UD-Q3_K_XL writes 83 tokens/s and reads 1,700; on the RTX 5090, 337-345 and 6,570-6,830.
• --cuda 12 (and a Pascal or Volta card, where setup picks CUDA 12 by itself) compiles strata-poly with a CUDA 12.x toolkit on your PC instead of stopping (#3). Built and run with CUDA 12.4 and 12.8 on Linux and CUDA 12.9 on Windows; no real Pascal card has run it yet.
• Answers come faster where most experts are outside VRAM. The expert cache keeps more of the card (a reserve of 256 MiB on an 8 GB card where it kept 512, and the cache grows back after another program held VRAM), a short prompt and the turns of a chat copy fewer experts before they start (first token of a 57-token prompt 0.5 s where 1.3), and a call of a few hundred tokens leaves a share of its experts to the CPU while the card copies the rest (a 150-token turn behind a 16,000-token prompt 0.43 s where 0.81 on Qwen3.6, 0.40 where 0.69 on Gemma 4). On the laptop Qwen3.6 UD-Q3_K_M writes a short chat at 94-100 tokens/s where 84-87 and reads at 1,690 where 1,500-1,620. The PC's RAM being tight for the model (the two largest files on a 32 GB laptop) keeps the larger reserve, which reads faster there.
• Fixes: a chat request with no messages is a 400 on every model (Gemma 4 and GLM answered it); GLM's and Gemma 4's thinking that ends on a quote no longer swallows the answer; a failed CUDA call no longer fails the request after it too; the server names the signal that ended the engine; a container with less memory than the model's files and 8 GB gets a warning at load, since a pod with 80 GiB and a 79 GB file read experts from the disk and was ended by the pod.
• For Claude Code: "fit_max_tokens": true in the run config, or every request is refused at a 32K context (it asks for 32,000 tokens of answer room). The FAQ has the three settings and their measured effect. The README's GIF is recorded again with this engine.
• A comparison with llama.cpp b11540 on the laptop, for Qwen3.6 and Tiel Coder, with the runs under bench/results.

How fast

Tokens per second, the first size of each model, with a short chat and a prompt of 30,000 to 32,000 tokens.

Model	Laptop: RTX 4070 Laptop (8 GB), 32 GB RAM	Workstation: RTX 5090 (32 GB)
Qwen3.6-35B-A3B UD-Q3_K_M	writes 94-100, reads 1,690	writes 394-395, reads 6,350-6,550
Gemma 4 26B-A4B UD-Q3_K_M	writes 92-93, reads 2,240	writes 353-356, reads 7,880-8,230
Ornith 1.5 APEX-I-COMPACT	writes 76-82, reads 1,620	writes 316-319, reads 6,830-6,860
Tiel Coder 35B-A3B UD-Q3_K_XL	writes 83, reads 1,700	writes 337-345, reads 6,570-6,830
GLM 5.3 Flash 3.0BIT	needs more RAM than it has	writes 59-66, reads 820 (a 29K-token prompt)

Every size and how it was measured: README and docs/BACKENDS.md.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh; the zip below is the ready-made engine for Windows, which the installer downloads and checks.

Limits

• The five added models are experimental, measured on two machines and a pair of RTX 3080s so far.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 compiles on your PC.
• GLM 5.3 Flash was not run on Windows by this project; a user runs it there with a 24 GB card and 128 GB of RAM (#10). It needs about 100 GB of RAM, and about 8 GB more than its file in a container.
• AMD cards work for Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.