0.1.45-poly.2

Faster long contexts for Qwen3.6 behind two switches, two opt-in speedups for GLM 5.3 Flash, the RX 6600 series in setup, and setup now takes your own GGUF through --gguf-dir without deleting anything.

What changed since 0.1.45-poly.1

• STRATA_FATTN=1 reads a long prompt's attention through llama.cpp's flash-attention kernel, and STRATA_FDEC=1 writes with a tensor-core attention kernel that reads the cache once. They are for Qwen3.6 and the models built on it (Ornith 1.5, Tiel Coder), on RTX 30 cards or newer. On an RTX 5090 with Qwen3.6 UD-Q4_K_M and a 96K prompt, decode went from 58 to 215 tokens/s with --kv q8_0 and from 86 to 189 with the default cache, and the prompt read from 6,145 to 8,175 tokens/s. On an 8 GB card, where most experts run on the CPU, decode stays about the same. Both are off unless set. By travysh (PR 50, PR 51; docs/DETAILS.md).
• GLM 5.3 Flash: --skip-weight F leaves out an expert outside VRAM by its router weight instead of its place, which on the RTX 5090 left out 17% more experts than --tail-skip 5 and wrote faster with guesses off (PR 34). --stage-read-mib 4096 lets a long prompt borrow VRAM from the expert cache while it is read, so each layer's experts go to the card once: with --stage-mib 1024, 8K to 32K prompts read at 1,146 to 1,223 tokens/s instead of 782 to 790, and decode afterwards is unchanged (PR 46). Both by Xiaomi2008, both off unless set.
• --gguf-dir takes your own GGUF: a fine-tune or another quantization of the same architecture, under the name of the closest size, is used as it is, with a warning (docs/FAQ.md, issue 54). Before, setup stopped on the file's size, and a file of the right size with another hash was deleted, also in your own folder; setup now never deletes a file in the --gguf-dir folder.
• Setup knows the RX 6600 / 6600 XT / 6650 XT (gfx1032) on Linux, unvalidated, after an owner ran Qwen3.6 on one (discussion 49).
• tools/strata_bench.py names an AMD card when ROCm came from AMD's wheels (no rocm-smi), instead of labelling the run "cpu". Found in the RX 6700 XT reports firetussin added to bench/results (PR 52).

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256. With the new switches off it gives the same greedy tokens as 0.1.45-poly.1's engine with Qwen3.6 and GLM 5.3 Flash on an RTX 5090 and with Qwen3.6 on an RTX 4070 laptop.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The eight added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container. A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.