0.1.45-poly.7

The strata-poly models keep less of your RAM at the same speed, a new opt-in setting about halves it again at some cost in speed, and a client that saves a session no longer ends the engine.

What changed since 0.1.45-poly.6

• The eleven models on strata-poly keep less of your RAM, with no setting. The engine no longer holds the parts of
  the model file that are already on the graphics card, and it drops cuBLAS's pages between requests. On an RTX 4070
  laptop (8 GB), Qwen3.6-35B-A3B UD-Q3_K_M after a 32,000-token prompt held 17.9 GiB with 0.1.45-poly.6 and holds
  13.1 GiB now, and writes as fast; under Linux it went from 17.8 to 12.6 GiB. Pinned memory went from 1.2 to 1.4 GiB
  down to 0.17. Reading a prompt needs 350 to 400 MiB less VRAM, which the expert cache takes (Qwen3.6 keeps 2,712
  experts in VRAM where 2,444).
• A parked conversation goes to a temporary file between requests instead of staying in the engine's memory: with
  a 32K conversation parked the engine held 12.58 GiB where 13.69 on the laptop, and the conversation came back in
  163 ms where 121. The image encoder on the CPU starts with the first picture and stops after 120 s without one,
  where it held 1.1 GiB from the server's start; the first picture after a pause waits 0.9 to 1.9 s for it.
• New and off by default: --ram-experts-mib 6144 holds 6 GiB of RAM for the experts outside VRAM, while it
  answers too, and leaves the others in the system's file cache, where other programs can take that memory. It
  writes slower, and we are still working on that. On the laptop with Qwen3.6 UD-Q3_K_M it held 6.5 to 7.3 GiB where
  11.7 to 14.3. Over three runs it wrote within 3% of the default in the first short chat, 3 to 8% slower after a
  32K prompt and 8 to 13% slower in a short chat after that. Under Linux (an RTX PRO 6000 held to about 8 GB) it
  held 7.0 to 7.7 GiB where 12.3 to 14.3 and wrote 7 to 14% slower. To use it, add "--ram-experts-mib", "6144" to
  the "args" list of strata-<model>.json and restart. The numbers are in the
  README, and how it works in
  docs/BACKENDS.md.
• The server answers a session save or restore (POST /slots/0?action=save with --slot-save-path) with 501 on
  the strata-poly models, which have no disk sessions, instead of ending the engine. A proxy that saved sessions
  restarted Tiel Coder over and over. Reported by maverde73
  (issue 114).
• STRATA_POLY_STREAM_STAGE=1, off unless set, copies the experts into the staging buffer with streaming stores.
  By vivekpokale21 (PR 109).
  The answers are the same with it and without it; on the RTX 4070 laptop it read and wrote as fast.
• New community results in
  docs/COMMUNITY_BENCHMARKS.md:
  an RX 7900 XTX with Clover Q4_K_M by brocasm
  (issue 115), a GTX 1650 4 GB with Clover and MiniCPM-V by
  LionelColaso (PR 111) and an
  RTX 3080 with Tiel Coder by HexhamAllstar
  (PR 113).

The engine changed, so the zip below is a new ready-made Windows engine. With the expert cache at 0.1.45-poly.6's
size (--stage-mib 2048 --stage-read-mib 0) it gives 0.1.45-poly.6's greedy tokens on an RTX 4070 laptop under
Windows with Qwen3.6, LFM2.5, Gemma 4, Ornith, Tiel Coder, MiniCPM-V and Clover, with STRATA_FATTN=0 STRATA_FDEC=0
and, for the Qwen3.6 family, with the defaults too, and with Qwen3.6 on an RTX PRO 6000 under Linux. Flint gave
them for a short prompt; its 3,000-token prompt was not run, since poly.6's engine holds its whole 18.5 GB file and
the laptop did not have the RAM free. With the defaults the cache holds more experts, the CPU and the card share the
work another way, and an answer can part from poly.6's where two tokens are close. With the experts held still,
--ram-experts-mib 6144 gives the same tokens as the default: Qwen3.6, LFM2.5, Gemma 4, Flint and Clover on the
laptop, Qwen3.6 under Linux. An install of 0.1.45-poly.6 updated to it with UPDATE.bat and kept its config as it
was. GLM 5.3 Flash, DeepSeek V4 Flash and Kolibri 1 were not run for this release.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The eleven added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
  A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• GLM 5.3 Flash, DeepSeek V4 Flash, Flint, Kolibri 1 and Clover have not been tried on an AMD card. On Windows, AMD
  cards have run one strata-poly model (Tiel Coder on an RX 9070 XT).
• --ram-experts-mib was measured with 6 to 12 GiB of experts outside VRAM, not with more.

Most of this project is the work of Strata's author and contributors.

Planned: SSD optimization for strata-poly, and a faster --ram-experts-mib.

0.1.45-poly.6

A new model, Clover-35B-A3B-Preview, IsValorum's APEX-I NanoPlus files, faster attention at long contexts for Qwen3.6 and the models built on it, and an opt-in router bias toward the experts in VRAM.

What changed since 0.1.45-poly.5

• Clover-35B-A3B-Preview by DiffuSpace, Qwen3.6-35B-A3B trained further to also write a block of tokens at once,
  runs on strata-poly: pick it in START-HERE or run ./setup.sh --family clover. Three sizes, 13 to 21 GB:
  maternion's Q3_K_M (the default), IsValorum's APEX-I-NanoPlus (the least RAM) and maternion's Q4_K_M. On an RTX 4070
  laptop (8 GB) the Q3_K_M writes at 68 to 73 tokens/s and reads a 32,000-token prompt at 1,560 to 1,700. It reads
  text only. Its own block guesses wrote slower than one token at a time even with every expert in VRAM, so they are
  off; STRATA_BLOCK_GUESS=1 turns them on. Its page is
  docs/models/clover.md.
• Files whose experts keep gate and up in two different types, as IsValorum's APEX-I NanoPlus files do, load now
  instead of stopping with "have different types". Flint's NanoPlus file picks llama.cpp's top token at 119 to 124 of
  128 greedy steps. Reported by Aleksisin
  (issue 103).
• Qwen3.6 and the models built on it (Ornith, Tiel Coder, MiniCPM-V, Clover) read prompts through llama.cpp's
  flash-attention kernel, and from 4,096 tokens of context write through a tensor-core decode kernel, on RTX 30
  cards and newer. Both were opt-in before. On an RTX PRO 6000 held to 12 GB at 100K of context, Qwen3.6 read 8,210
  tokens/s instead of 5,837 and wrote 230 instead of 106; on an RTX 4060 laptop at 118K,
  GodrezJr2 measured 1,318 instead of 814 reading and 51.0 instead of 32.1 writing
  (PR 108). Where two tokens are almost equally likely, an answer can
  now part from the one 0.1.45-poly.5 gave. STRATA_FATTN=0 and STRATA_FDEC=0 turn each one off.
• --cache-bias F, off unless set and for those same models only, makes a token that comes a few at a time
  lean toward the experts already in VRAM. At 12 GB with F = 0.5 the top token stayed the same at 121 to 126 of 128
  steps and writing went from 196 to 234 tokens/s. By drakarah
  (PR 104, issue 81).
• Changes that UPDATE.bat or ./update.sh brought before this release: when the context cuts a reply short, the
  thinking still closes in time to answer; tools/histogram_profile.py makes an expert profile from counted routing
  (festeraeb, PR 101); the RX 5700 XT
  (gfx1010) installs as an unvalidated RDNA1 card (firetussin,
  PR 102).

The engine changed, so the zip below is a new ready-made Windows engine. With STRATA_FATTN=0 and STRATA_FDEC=0
it gives 0.1.45-poly.5's greedy tokens with Qwen3.6, LFM2.5, Gemma 4, Ornith, Tiel Coder, MiniCPM-V and Flint on an
RTX 4070 laptop under Windows, and with Qwen3.6 on an RTX PRO 6000 under Linux. With the defaults the short prompts
gave the same tokens and the long ones parted where two tokens were close: on the RTX PRO 6000, after a 4,833-token
prompt, Qwen3.6 picked 0.1.45-poly.5's top token at 124 of 128 steps, and at the other four that token was in its
top three. Clover picks llama.cpp's top token at 125 to 127 of 128 steps on the laptop. An install of 0.1.45-poly.5
updated to it with UPDATE.bat and kept its config as it was. GLM 5.3 Flash, DeepSeek V4 Flash and Kolibri 1 were not
run for this release.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The eleven added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
  A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• GLM 5.3 Flash, DeepSeek V4 Flash, Flint, Kolibri 1 and Clover have not been tried on an AMD card. On Windows, AMD
  cards have run one strata-poly model (Tiel Coder on an RX 9070 XT).

Most of this project is the work of Strata's author and contributors.

Planned: SSD optimization for strata-poly.

0.1.45-poly.5

A new model, Kolibri 1, Gemma 4's QAT file as a size of its own, and faster writing on AMD cards.

What changed since 0.1.45-poly.4

• Kolibri 1 by Aleph Alpha, a model of 19,200 experts that writes German as well as English, runs on strata-poly:
  pick it in START-HERE or run ./setup.sh --family kolibri. Hob-forge's Q4_K_M, Q2_K and Q3_K_M files are 29 to
  48 GB. On an RTX 5090 the Q2_K writes at 189 to 192 tokens/s and reads a 31,676-token prompt at 4,980 to 5,070. It
  has not been run on an 8 GB laptop or on an AMD card yet. By JLSoftwareRL
  (PR 89); its page is
  docs/models/kolibri.md.
• Gemma 4's quantization-aware trained file from Unsloth is a size of its own: --model QAT-UD-Q4_K_XL (15 GB, with
  its own draft model). On the RTX 5090 it writes at 356 to 371 tokens/s and reads a prompt of about 31,000 tokens at
  8,460 to 8,680. It needed no engine change, so UPDATE.bat or ./update.sh brought it before this release.
• On AMD cards the cached experts give a short row a warp of its own, so writing is faster: 18 to 33% on an RX 7800 XT
  by the author's measurements. On by default on AMD. On NVIDIA it stays off; STRATA_SLOTS_WARP=1 turns it on, and
  on an RTX 5090 that wrote about 4% faster with every expert in VRAM and the same otherwise. By
  usmanliaqat97 (PR 94).
• STRATA_HIP_VALUE_SPLIT=1, AMD only and off unless set, splits the decode attention's last step for Qwen3.6 and
  its variants once the context passes 12,288 tokens. On an RX 9070 XT: 48.5 to 52.3 tokens/s at 32K, 36.5 to 44.6
  at 98K. By leakedd (PR 85,
  PR 86).
• STRATA_REQUEST_LINES=1 prints a line per request for the models of PolyStrata's engine too, with the MTP drafts
  accepted. By vehystrix (PR 90).
• strata-poly compiles for Pascal sm_60 (Tesla P100) again with CUDA 12. By festeraeb
  (PR 87).
• Setup changes that UPDATE.bat or ./update.sh brought before this release: --custom-gguf installs a GGUF you
  already have (vehystrix, PR 83); Gemma 4, LFM2.5 and Tiel Coder
  no longer warn that they are untried on AMD, since people have run them on an RX 6700 XT and an RX 7800 XT.

The engine changed, so the zip below is a new ready-made Windows engine. It gives 0.1.45-poly.4's greedy tokens with
Qwen3.6 (with and without --kv q8_0), Gemma 4 (with and without --kv int8), Gemma 4 QAT, LFM2.5, MiniCPM-V,
Flint, GLM 5.3 Flash and DeepSeek V4 Flash on an RTX 5090 under Linux, and Kolibri 1 gives PR 89's own tokens there. On an RTX
4070 laptop under Windows it gives 0.1.45-poly.4's tokens with Gemma 4, and an install of 0.1.45-poly.4 updated to it
with UPDATE.bat and kept its config as it was.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The ten added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
  A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• GLM 5.3 Flash, DeepSeek V4 Flash, Flint and Kolibri 1 have not been tried on an AMD card. On Windows, AMD cards have
  run one strata-poly model (Tiel Coder on an RX 9070 XT).

Most of this project is the work of Strata's author and contributors.

Planned: SSD optimization for strata-poly.

0.1.45-poly.4

A faster first answer after a cold start on Windows, a conversation cache sized from the RAM you have free, and a
fix for Gemma 4 answers that came back empty.

What changed since 0.1.45-poly.3

• On Windows, an answer asked right after the model loads no longer comes at half speed when the model's file is not
  in the file cache yet (the first start after a reboot, or a start after other programs used the RAM). The engine
  brought the experts into memory one page fault at a time, about 1.3 GB/s from a laptop's SSD; a second thread now
  reads them through the file cache beside it, at about 3.2 GB/s. On an RTX 4070 laptop with Qwen3.6 UD-Q3_K_M the
  experts are in after 3.5 to 5 s instead of 10, and an answer asked right after the load is written at 62 to 80
  tokens/s instead of 46 to 48. When the file cache already holds the file, that first answer is about 3% slower and
  the ones after it are the same. Linux is unchanged.
• --conversation-cache-mib auto sizes the conversation cache from the RAM free once the model is loaded: 40% of
  what is left above the 2,560 MiB the cache always leaves free, at most 8,192 MiB, and off under 1,024. It is the
  rule of the Strata project's engine. Setup writes auto when you answer yes
  to the conversation cache. A number, 0 or no option mean what they meant before, so a config setup wrote earlier
  keeps its 8192.
• Gemma 4 with a thinking budget could answer nothing: the server closed its thinking with </think>, which Gemma
  reads as text inside its thought, so it thought on to max_tokens. It closes it with <channel|> now. The server
  sets such a budget for any request with max_tokens of 1024 or more, so a long thought was enough. Reported and
  checked on an RX 6700 XT by mkw432
  (discussion 60): 2 of 5 replies empty before, 0 of 5 after.
  It is a server change, so UPDATE.bat or ./update.sh had it before this release.
• STRATA_FATTN=1 and STRATA_FDEC=1, the opt-in tensor-core attention for prompts and for writing, take Ornith 1.5
  122B's 16 query heads per KV head. Off unless set. By ersensari
  (PR 63).

The engine changed, so the zip below is a new ready-made Windows engine. It gives 0.1.45-poly.3's greedy tokens with
Qwen3.6, LFM2.5, Gemma 4, Ornith 1.5, Tiel Coder, MiniCPM-V and Flint on an RTX 4070 laptop (with and without the
MTP drafts), and with Qwen3.6, LFM2.5, MiniCPM-V, GLM 5.3 Flash and DeepSeek V4 Flash on an RTX 5090 under Linux.
An install of 0.1.45-poly.3 updated to it with UPDATE.bat and kept its config as it was.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The nine added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
  A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• On Windows, AMD cards have run one strata-poly model on one card (Tiel Coder on an RX 9070 XT); the others are
  untried there.

Most of this project is the work of Strata's author and contributors.

Planned: SSD optimization for strata-poly.

0.1.45-poly.3

Flint 40B-A2.5B, Recursive's model for coding agents, faster than llama.cpp on the two machines it was tried on;
Gemma 4, Qwen3.6 and the other strata-poly models on AMD cards under Windows; and fixes to setup.

What changed since 0.1.45-poly.2

• Flint 40B-A2.5B by Recursive: ./setup.sh --family flint, or pick it in START-HERE.bat. Four sizes from
  Unsloth: UD-Q4_K_XL, UD-Q3_K_XL, UD-Q6_K_XL and Q8_0. Beside an 8 GB card setup takes UD-Q3_K_XL below about 40 GB
  of RAM: on a 32 GB laptop UD-Q4_K_XL's file did not stay in RAM. On an RTX 4070 laptop UD-Q3_K_XL reads a prompt
  of 29,000 tokens at 2,000 tokens/s and writes a short chat at 61 to 63; llama.cpp a37cc46104 read it at 390 and
  wrote after it at 35 there. On an RTX 5090 UD-Q4_K_XL reads it at 12,900 tokens/s and writes at 251; llama-bench
  reads 29,816 tokens at 12,470 and writes at 222. Asked for by hum-ma, HardeNNN and MaverickBlue
  (issue 16). Details: docs/FLINT.md.
• The product of a prompt batch with a router's 128 or 256 outputs goes in tiles, with the same bits as before:
  Flint's router takes 94 ms of a 29,816-token read on the 5090 instead of 241. Qwen3.6, Ornith, Tiel Coder,
  MiniCPM-V, DeepSeek V4 and Gemma 4 take the same kernel.
• Setup compiles strata-poly for an AMD card on Windows instead of refusing the models there: AMD's ROCm wheels, the
  Visual Studio C++ build tools and ROCm's compiler, with the ROCm libraries of the ready-made main engine. On an RX
  9070 XT with Tiel Coder UD-Q3_K_XL it read a 7,376-token prompt at 2,750 tokens/s and wrote at 55 to 75. That is
  the one card and model run so far; setup says so with a warning. By berkkarahan
  (PR 53).
• Your own GGUF: a whole file of the right architecture put in setup's own models folder now gets the way to run it
  (its own folder and --gguf-dir) instead of "delete it and run setup again". A file whose experts' gate and up
  have different types, which the engine cannot run, is refused before the install with the layer and the types,
  instead of stopping the engine at its first start (issue 54).
• --check-batch gives the attention caches their VRAM at load, so the batched phase no longer moves experts to the
  CPU and the comparison no longer reports differences that come from that. Found by
  donschen (issue 55).
• STRATA_GDN_MMA=1 reads the linear-attention layers of Qwen3.6, Ornith 1.5, Tiel Coder and MiniCPM-V with
  llama.cpp's tensor-core kernel. A 32K prompt read about 12% faster with Qwen3.6 and MiniCPM-V on an RTX 5090 and
  12 to 18% faster with Qwen3.6 and Tiel Coder on an RTX 4070 laptop; writing is unchanged. Off unless set; RTX 30
  cards and newer. By vivekpokale21 (PR 65).
• The chat page can regenerate an answer and step through its variants (arrows and an n/m counter, kept with the
  chat). By Ziomd (PR 74).
• --kv int8 and --kv int8k keep Gemma 4's attention caches in int8 (int8k: the keys only), opt-in; fp16
  stays the default. By smashdog (PR 59).
• GLM 5.3 Flash no longer fails at its first token on a Tesla V100, or on an RTX 20 card with a CUDA 12 build: one
  of its kernels asked for more registers than a block of 1,024 threads can have. Found by
  llollaalloll (issue 73).
• Setup goes on in a PolyStrata folder that was moved after it was installed: when the cmake in .venv still names
  the old place, setup installs it again instead of stopping at the engine build with "Fatal error in launcher"
  (issue 78). On AMD cards setup now runs ROCm's rocm_sdk through
  the venv's Python for the same reason: its rocm-sdk launcher named the old place too.
• Setup no longer stops with "unknown target 'strata-poly'" when it builds the engine again in a folder it built
  before and finds nvcc as nvcc.EXE where the folder has nvcc.exe: CMake took that for another compiler and
  dropped its settings.
• On Linux with an AMD card, the web page's GPU tiles of a strata-poly model said "not readable": setup now writes
  "backend": "hip" in those configs on Linux too, so run setup again to get it. Found by
  mkw432 (PR 71).
• A GLM reply now stops at <|user|> even when the GGUF names no end of turn (REAP50-Q3_K_M ran on to max_tokens),
  reported by whp199 (issue 70).
• Docs: the RX 6600 series runs setup's own engine without HSA_OVERRIDE_GFX_VERSION, and an override left from an
  older guide is what makes it fail with "device kernel image is invalid"
  (discussion 49); community results for Tiel Coder and
  MiniCPM-V on an RTX 4070 Ti SUPER, four models on an RX 6700 XT with a Ryzen 5 5600X (mkw432,
  PR 71), and Ornith 1.5 122B on an RX 7900 XTX with an R9700
  (Danielkgr).

The engine changed, so the zip below is a new ready-made Windows engine. It gives 0.1.45-poly.2's greedy tokens
with Qwen3.6, LFM2.5, Gemma 4, Ornith 1.5, Tiel Coder and MiniCPM-V on an RTX 4070 laptop (with and without the
MTP drafts), and with Qwen3.6, LFM2.5, MiniCPM-V, GLM 5.3 Flash and DeepSeek V4 Flash on an RTX 5090, and it
reads prompts and writes at least as fast as poly.2's engine there.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The nine added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
  A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• On Windows, AMD cards have run one strata-poly model on one card (Tiel Coder on an RX 9070 XT); the others are
  untried there.

Most of this project is the work of Strata's author and contributors.

Being worked on now: reading experts from the SSD faster when a model's file does not fit in RAM, as with Flint's UD-Q6_K_XL and Q8_0 on a 32 GB laptop.

0.1.45-poly.2

Faster long contexts for Qwen3.6 behind two switches, two opt-in speedups for GLM 5.3 Flash, the RX 6600 series in setup, and setup now takes your own GGUF through --gguf-dir without deleting anything.

What changed since 0.1.45-poly.1

• STRATA_FATTN=1 reads a long prompt's attention through llama.cpp's flash-attention kernel, and STRATA_FDEC=1 writes with a tensor-core attention kernel that reads the cache once. They are for Qwen3.6 and the models built on it (Ornith 1.5, Tiel Coder), on RTX 30 cards or newer. On an RTX 5090 with Qwen3.6 UD-Q4_K_M and a 96K prompt, decode went from 58 to 215 tokens/s with --kv q8_0 and from 86 to 189 with the default cache, and the prompt read from 6,145 to 8,175 tokens/s. On an 8 GB card, where most experts run on the CPU, decode stays about the same. Both are off unless set. By travysh (PR 50, PR 51; docs/DETAILS.md).
• GLM 5.3 Flash: --skip-weight F leaves out an expert outside VRAM by its router weight instead of its place, which on the RTX 5090 left out 17% more experts than --tail-skip 5 and wrote faster with guesses off (PR 34). --stage-read-mib 4096 lets a long prompt borrow VRAM from the expert cache while it is read, so each layer's experts go to the card once: with --stage-mib 1024, 8K to 32K prompts read at 1,146 to 1,223 tokens/s instead of 782 to 790, and decode afterwards is unchanged (PR 46). Both by Xiaomi2008, both off unless set.
• --gguf-dir takes your own GGUF: a fine-tune or another quantization of the same architecture, under the name of the closest size, is used as it is, with a warning (docs/FAQ.md, issue 54). Before, setup stopped on the file's size, and a file of the right size with another hash was deleted, also in your own folder; setup now never deletes a file in the --gguf-dir folder.
• Setup knows the RX 6600 / 6600 XT / 6650 XT (gfx1032) on Linux, unvalidated, after an owner ran Qwen3.6 on one (discussion 49).
• tools/strata_bench.py names an AMD card when ROCm came from AMD's wheels (no rocm-smi), instead of labelling the run "cpu". Found in the RX 6700 XT reports firetussin added to bench/results (PR 52).

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256. With the new switches off it gives the same greedy tokens as 0.1.45-poly.1's engine with Qwen3.6 and GLM 5.3 Flash on an RTX 5090 and with Qwen3.6 on an RTX 4070 laptop.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The eight added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container. A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.

0.1.45-poly.1

The Strata project's 0.1.45 is in: Qwen3.6 on Strata's engine reads pictures and answers several requests at once, and a model switch in the web app now runs the new model with its own settings.

What changed since 0.1.44.1-poly.6

• The Strata project's 0.1.45 is merged. Qwen3.8-Flash-Next, which runs on Strata's own engine, takes everything in it, from the faster decode kernels to the opt-in switches for servers with many users. Qwen3.6 on Strata's engine (--family qwen36-strata) now reads pictures (--vision) and answers several requests at once (--parallel), with faster decode kernels (docs/QWEN36.md).
• For every model, two things from 0.1.45's server: a long chat's prompt is encoded again only from where it differs from the last one, and when a request allows 1,024 tokens or more and sets no thinking budget, thinking is closed early enough to leave a quarter of max_tokens for the answer. "answer_room": false in strata-<model>.json turns the second one off. Strata's server also gives Qwen3.8-Flash-Next its model card's sampling when a request names none; the models on strata-poly keep their own defaults.
• A model switch in the web app runs the new model with its own strata-<model>.json: before, every key but the sampling, the aliases and the GPU stayed as the first model set them, and Ctrl+C after a switch closed the wrong engine. By arbitropy (PR 39, issue 37).
• AMD cards on Linux: the expert cache gives VRAM back to other programs and grows again when they free it, instead of only shrinking until a restart (PR 28, issue 13), and --kv q8_0 runs on AMD cards (PR 29). Both by donschen, measured on an RX 9070 XT.
• STRATA_FREEZE_EXPERTS=1 holds the split of the experts between the card and the CPU still, so the same request gives the same greedy answer bit for bit; with guesses on, add --window 1 for that everywhere (PR 32). The load now writes VRAM on the engine's own stream (PR 31). Both by smashdog; Tako88 checked the first on an AMD card.
• docs/DEEPSEEK.md: what a user measured with DeepSeek V4 Flash files larger than the RAM (issue 21).

The engine's shared kernels changed with the merge, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256. It gives the same greedy tokens as 0.1.44.1-poly.6's engine on an RTX 5090 with GLM 5.3 Flash, DeepSeek V4 Flash, Qwen3.6, LFM2.5 and MiniCPM-V, and on an RTX 4070 laptop with Gemma 4, and several requests at once still give each the tokens it gets alone.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The eight added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container. A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.

0.1.44.1-poly.6

The eight added models answer several requests at once, and DeepSeek V4 Flash's MXFP4 experts run on the card without a converted copy in RAM.

What changed since 0.1.44.1-poly.5

• "parallel": N in strata-<model>.json, or --parallel N when you run setup, now works for GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V, up to 8 (issue 22). Before, strata-poly answered one request at a time and the server queued the rest. The server drives it as it drives the Strata project's engine: a request alone writes with guesses as before, and when a second comes the two write together in batch slots, each with its own position, recurrent state and attention rows, while the weights and the experts they share are read once for both. A long prompt being read gives way to the slots between its pieces, so a chat does not stall while someone else pastes a long document. On an RTX 5090 with Qwen3.6 UD-Q4_K_M, four 300-token chats at once came at 428 to 430 tokens/s in all, where one alone writes at 312 to 333. On an 8 GB laptop, where most experts run on the CPU, three Qwen3.6 UD-Q3_K_M chats at once came at 105 tokens/s together and at 79 one after the other. With the expert cache fixed and no guesses, each answer is the one it would get alone: strata-poly --check-batch compares them token by token, every token matched for all five architectures, and three chats through the server gave the same text at once as alone. With the default settings two runs of a chat already differ after a few dozen words, together or alone. How it works and how to check it: docs/BATCHING.md.
• DeepSeek V4 Flash's MXFP4 experts are multiplied on the card now. Before, the engine converted the four MXFP4 layers' down experts to Q5_0 at load, into RAM that cannot be paged back to the file: 6 GB with UD-IQ2_XXS, 1.38 GiB a layer with UD-IQ4_XS, which mattered on a PC whose RAM is smaller than the file. On the 5090 the engine's own RAM went from 5,959 MiB to 341 with UD-IQ2_XXS, a 24K-token prompt read at 1,091 tokens/s where 1,000 before, and writing speed is the same. On RTX 50 cards llama.cpp's FP4 tiles quantize the prompt's rows to FP4, as llama.cpp does, so greedy text can differ from 0.1.44.1-poly.5 after a few tokens.
• The README's speed chart has all eight added models, and there is an issue form for model requests.

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh, and START-HERE.bat --setup --parallel 3 turns on three requests at once.

Limits

• The eight added models are experimental.
• Requests in batch slots write without guesses, and --batch-groups and the other batching options of the Strata project's engine are not there yet.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.

0.1.44.1-poly.5

MiniCPM-V 4.7 35B-A3B reads pictures on PolyStrata, the Qwen3.6 family can keep its attention cache in 8 bits, and a model can keep the thinking of earlier turns.

What changed since 0.1.44.1-poly.4

• MiniCPM-V 4.7 35B-A3B by OpenBMB is the ninth model, brought in by iMilnb (PR 20): ./setup.sh --family minicpmv installs one of three sizes, Q4_K_M, mix_q3gud_im or Q6_K (16 to 29 GB). It has Qwen3.6-35B-A3B's architecture and its own weights, made to read pictures. A large picture is read as an overview and a few tiles, and on four test pictures, from a 64x64 icon to a 2520x1620 chart, it read the same numbers and labels as llama.cpp's own llama-mtmd-cli. Its file has no draft block, so it writes one token at a time: with Q4_K_M an RTX 4070 laptop (8 GB) writes a short chat at 64 tokens/s and reads a 32K-token prompt at 1,150; an RTX 5090 writes at 308 to 311 and reads at 7,910 to 7,950. Its license is Apache 2.0.
• --kv q8_0 for Qwen3.6, Ornith, Tiel Coder and MiniCPM-V, by vpkoskinen (PR 23): the attention cache in ggml's Q8_0 blocks, about half its VRAM, which the expert cache can use at long contexts. It is off unless you add it to the run config's args; without it the engine gives the same text as before. On the 5090 it wrote 2% to 6% slower after an 8K prompt; on the 8 GB laptop there was no difference to measure.
• "preserve_thinking": true in strata-<model>.json, or "chat_template_kwargs": {"preserve_thinking": true} in a request (issue 25): Qwen3.6's template, which Ornith and Tiel Coder share, then keeps the thinking of earlier turns in the prompt instead of dropping it, and the server remembers the thinking of the answers it wrote, so it works with apps that send an answer back without it (the chat page, Open WebUI, VS Code's Copilot). In a test where the model thinks of two 20-digit numbers and says one, with the app dropping the thinking, it gave the other exactly when asked later, and made one up without the setting.
• The picture encoder builds from llama.cpp 69f201a, which MiniCPM-V needs; the two engines stay on the Strata project's 3cf03257.

The engine and the picture encoder changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

How fast

The README has MiniCPM-V's tables for both PCs, next to the other models'.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh, and START-HERE.bat --setup --family minicpmv adds MiniCPM-V.

Limits

• The eight added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• --kv q8_0 is for the qwen35moe models only; the others refuse it at start. It has not been tried on an AMD card, where the engine refuses it.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.

0.1.44.1-poly.4

strata-poly gives the expert cache back the VRAM a long request took, and the server can drop a chat's oldest turns when it no longer fits.

What changed since 0.1.44.1-poly.3

• The expert cache gets its VRAM back after a long request. The attention caches grow as a conversation does, and until now they kept that VRAM for as long as the server ran, so every request after one long read had fewer experts on the card. Now the rows that neither the current request nor the one before it reaches go back to the expert cache. GLM 5.3 Flash on an RTX 5090 with 8 GB kept free, after a 287K-token request: the requests after it wrote at 64 to 65 tokens/s before and 70 to 71 now, with 1,631 experts in VRAM instead of 1,192. A context set to 1M that is not filled costs nothing either way: GLM starts with 1,684 experts at 32K and 1,672 at 1M (issue 10).
• "drop_oldest": true in strata-<model>.json (off by default): when a chat no longer fits the context, the server leaves out its oldest turns until there is room for the answer, instead of refusing the request. The system prompt and the last message stay (issue 17, FAQ).
• DeepSeek V4 Flash runs on Windows, by a user's report: UD-IQ2_XXS on an RTX 5090 D v2 (24 GB) with 128 GB of RAM, 29 to 36 tokens/s, and a filled 1M context read with every planted fact found (issue 21).

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The seven added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.