0.1.44.1-poly.4

strata-poly gives the expert cache back the VRAM a long request took, and the server can drop a chat's oldest turns when it no longer fits.

What changed since 0.1.44.1-poly.3

• The expert cache gets its VRAM back after a long request. The attention caches grow as a conversation does, and until now they kept that VRAM for as long as the server ran, so every request after one long read had fewer experts on the card. Now the rows that neither the current request nor the one before it reaches go back to the expert cache. GLM 5.3 Flash on an RTX 5090 with 8 GB kept free, after a 287K-token request: the requests after it wrote at 64 to 65 tokens/s before and 70 to 71 now, with 1,631 experts in VRAM instead of 1,192. A context set to 1M that is not filled costs nothing either way: GLM starts with 1,684 experts at 32K and 1,672 at 1M (issue 10).
• "drop_oldest": true in strata-<model>.json (off by default): when a chat no longer fits the context, the server leaves out its oldest turns until there is room for the answer, instead of refusing the request. The system prompt and the last message stay (issue 17, FAQ).
• DeepSeek V4 Flash runs on Windows, by a user's report: UD-IQ2_XXS on an RTX 5090 D v2 (24 GB) with 128 GB of RAM, 29 to 36 tokens/s, and a filled 1M context read with every planted fact found (issue 21).

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The seven added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.