0.1.45-poly.8

strata-poly runs on Apple Silicon Macs, and a PC whose RAM cannot hold the experts outside VRAM now keeps the most used ones there and reads the rest from the SSD.

What changed since 0.1.45-poly.7

• Qwen3.6 and LFM2.5 run on a Mac, as a beta, on a Metal port of strata-poly: ./setup.sh --family qwen36 or
  --family lfm25. The port is the Metal part of PR 135 by
  berkkarahan. On top of it a prompt runs on the GPU's matrix units (macOS 26 and
  newer) and a decoding token's attention is split among many threadgroups. On an M5 Ultra Mac Studio, Qwen3.6
  UD-Q2_K_XL wrote 134 tokens/s in a short chat and read a 3,768-token prompt at 2,978 tokens/s; LFM2.5 UD-Q3_K_XL
  wrote 153 tokens/s and read a 3,680-token prompt at 4,294 tokens/s. Setup also offers GLM, DeepSeek, Ornith,
  Gemma 4, Tiel Coder and MiniCPM-V on a Mac, but they have not been run there with their real weights. How to set it
  up and the numbers: docs/MAC.md.
• When the experts outside VRAM do not fit in the free RAM, the engine keeps the most used ones in a budget of that
  RAM less 4 GiB and reads the others from the SSD, and says so when it starts. Before, it mapped them all and the
  system read them back a page fault at a time. Under Linux on an RTX PRO 6000 held to about 8 GB, with the engine
  held to 12 GB of RAM, Qwen3.6 UD-Q3_K_M wrote 57-60 / 77-85 / 131-139 tokens/s (first chat / after a 24K prompt /
  the chat after it) where 51 / 75-78 / 94-98. A container's memory limit now counts as the free RAM.
  "--ram-experts-mib", "-1" keeps the old behavior.
• --lock-experts, off by default, locks the experts outside VRAM in RAM so the system cannot hand their memory to
  other programs (issue 79).
• Without --threads the engine now leaves 30% of the CPUs to other programs. Before, it took every CPU and kept
  them 90% busy while it wrote; under Linux 22 threads of 32 wrote as fast.
• Ornith 1.5 122B in setup (--family ornith122, UD-Q2_K_XL to Q8_0), by
  berkkarahan (PR 64).
• The web app: edit a user message and get a new answer to it, by Ziomd
  (PR 75); a Requests tab for the API monitor
  (issue 142).
• "truncate_middle": true in a model's config keeps the start and end of a prompt that does not fit the context,
  by abienias97 (PR 138).
• Gemma 4 with --batch no longer frees slots that are still in use, by
  jonasmanuel (PR 139).
  The Responses API's last stream event carries timings, by vehystrix
  (PR 118).
• AMD: START-HERE.bat --backend hip --build compiles the HIP engine on Windows, by
  berkkarahan (PR 57);
  setup knows the RX 5600 XT / 5700 / 5700 XT (PR 137) and
  the RX 6600 / 6600 XT / 6650 XT (issue 121); two AMD
  cards can share a model (PR 117, by
  imaginitivename); a ROCm Docker image, by
  jintakhan (PR 151).
• Opt-in, off unless asked: --kv q8_0 for Flint and LFM2.5, by smashdog
  (PR 152); --vram-pool, split-key decode attention and
  FP32 prompt attention on HIP, the SDWA dot product on gfx1010, and size classes for the staged experts' tiles, by
  drakarah (PR 143,
  145,
  147,
  140,
  149); STRATA_NO_SPILL=1 keeps parked conversations in
  RAM, by vehystrix (PR 119).
• The README in Turkish, by enderkus
  (PR 123). The engine configures without the tests folder
  (issue 124). New community results in
  docs/COMMUNITY_BENCHMARKS.md
  (PR 127,
  131,
  134).

The engine changed, so the zip below is a new ready-made Windows engine. It was built from this release's source
but not run on Windows for this release. Under Linux on an RTX PRO 6000 held to about 9 GB, the same source gives
0.1.45-poly.7's greedy tokens for a short prompt and a long one with Qwen3.6 (with STRATA_FATTN=0 STRATA_FDEC=0
and with the defaults), Flint, Tiel Coder and Clover, the expert cache at poly.7's size. On the M5 Ultra the 73
Metal tests pass, and Qwen3.6 and LFM2.5 answer prompts of 28 to 7,205 tokens. Of the setup, server and tools tests,
2,084 pass under Linux; the other one fails on the test machine's host name, as it would have with poly.7. LFM2.5, Gemma 4, Ornith, MiniCPM-V, GLM 5.3 Flash, DeepSeek V4 Flash and Kolibri 1 were not run on
an NVIDIA card for this release, and the AMD changes were not built for an AMD card.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux or a Mac. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The strata-poly models are experimental, and on a Mac they are a beta.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12, AMD cards and Macs compile on your computer.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• GLM 5.3 Flash, DeepSeek V4 Flash, Flint, Kolibri 1 and Clover have not been tried on an AMD card.

Most of this project is the work of Strata's author and contributors.