0.1.45-poly.4

A faster first answer after a cold start on Windows, a conversation cache sized from the RAM you have free, and a
fix for Gemma 4 answers that came back empty.

What changed since 0.1.45-poly.3

• On Windows, an answer asked right after the model loads no longer comes at half speed when the model's file is not
  in the file cache yet (the first start after a reboot, or a start after other programs used the RAM). The engine
  brought the experts into memory one page fault at a time, about 1.3 GB/s from a laptop's SSD; a second thread now
  reads them through the file cache beside it, at about 3.2 GB/s. On an RTX 4070 laptop with Qwen3.6 UD-Q3_K_M the
  experts are in after 3.5 to 5 s instead of 10, and an answer asked right after the load is written at 62 to 80
  tokens/s instead of 46 to 48. When the file cache already holds the file, that first answer is about 3% slower and
  the ones after it are the same. Linux is unchanged.
• --conversation-cache-mib auto sizes the conversation cache from the RAM free once the model is loaded: 40% of
  what is left above the 2,560 MiB the cache always leaves free, at most 8,192 MiB, and off under 1,024. It is the
  rule of the Strata project's engine. Setup writes auto when you answer yes
  to the conversation cache. A number, 0 or no option mean what they meant before, so a config setup wrote earlier
  keeps its 8192.
• Gemma 4 with a thinking budget could answer nothing: the server closed its thinking with </think>, which Gemma
  reads as text inside its thought, so it thought on to max_tokens. It closes it with <channel|> now. The server
  sets such a budget for any request with max_tokens of 1024 or more, so a long thought was enough. Reported and
  checked on an RX 6700 XT by mkw432
  (discussion 60): 2 of 5 replies empty before, 0 of 5 after.
  It is a server change, so UPDATE.bat or ./update.sh had it before this release.
• STRATA_FATTN=1 and STRATA_FDEC=1, the opt-in tensor-core attention for prompts and for writing, take Ornith 1.5
  122B's 16 query heads per KV head. Off unless set. By ersensari
  (PR 63).

The engine changed, so the zip below is a new ready-made Windows engine. It gives 0.1.45-poly.3's greedy tokens with
Qwen3.6, LFM2.5, Gemma 4, Ornith 1.5, Tiel Coder, MiniCPM-V and Flint on an RTX 4070 laptop (with and without the
MTP drafts), and with Qwen3.6, LFM2.5, MiniCPM-V, GLM 5.3 Flash and DeepSeek V4 Flash on an RTX 5090 under Linux.
An install of 0.1.45-poly.3 updated to it with UPDATE.bat and kept its config as it was.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The nine added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
  A DeepSeek file larger than the RAM runs several times slower than llama.cpp with the same file.
• On Windows, AMD cards have run one strata-poly model on one card (Tiel Coder on an RX 9070 XT); the others are
  untried there.

Most of this project is the work of Strata's author and contributors.

Planned: SSD optimization for strata-poly.