0.1.45-poly.9

Answers written together can guess ahead the way a single answer does, strata-poly reads NVFP4 GGUFs, and a prompt with a picture reads faster on a small card.

What changed since 0.1.45-poly.8

• --batch-mtp, off by default: with "parallel" above 1 and a model that carries a draft block (Qwen3.6, Ornith
  1.5, Tiel Coder), the answers written together each guess one token a step and keep it when the model picks the
  same. Each answer is the one it gets alone. With Qwen3.6 on an RTX PRO 6000, two answers together wrote 412-472
  tokens/s instead of 295-333 with every expert on the card, 160-164 instead of 144-154 with the card held to 9 GB,
  and three answers 442-446 instead of 428-429; on an M5 Ultra two answers wrote 173 instead of 140. Add
  "--batch-mtp" to the model's "args" to turn it on. The idea is
  Basalt's. Details:
  docs/BATCHING.md.
• strata-poly reads NVFP4 GGUFs, including files converted from NVIDIA's checkpoints, which keep a global scale
  beside each matrix. On an RTX 50 card STRATA_PREFILL_NVFP4=w4a4 reads the prompt on the FP4 tensor cores; it is
  off by default because it was no faster on Qwen3.6, 10% faster on LFM2.5, and moves the answers more. Setup does
  not offer an NVFP4 file. Numbers:
  docs/NVFP4.md.
• A prompt with a picture reads its text in larger parts, so the experts outside VRAM are copied to the card once
  for each part instead of once for every 1,024 tokens, by smashdog
  (PR 155). With Qwen3.6 on an RTX PRO 6000 held to 8 GB,
  18,256 tokens around a picture read at 2,401 tokens/s instead of 1,542-1,595. A prompt without a picture is read
  as before.
• New community results in
  docs/COMMUNITY_BENCHMARKS.md:
  Qwen3.6 on an RTX 5060 Ti (issue 153) and Flint on an
  RX 7900 XTX, its first run on an AMD card, by dontaskmeformyname
  (PR 156).

The engine changed, so the zip below is a new ready-made Windows engine. It was built from this release's source
but not run on Windows for this release. Under Linux on an RTX PRO 6000 held to about 9 GB, the same source gives
0.1.45-poly.8's greedy tokens for a short prompt and a long one with Qwen3.6 (with STRATA_FATTN=0 STRATA_FDEC=0
and with the defaults), Flint, Tiel Coder and Clover, with the same experts in VRAM; with --batch-mtp, two prompts
written together get the tokens each gets alone. On the M5 Ultra the 74 Metal tests pass. Of the setup, server and
tools tests, 2,084 pass under Linux; the other one fails on the test machine's host name, as it did with poly.8.
LFM2.5, Gemma 4, Ornith, MiniCPM-V, GLM 5.3 Flash, DeepSeek V4 Flash and Kolibri 1 were not run on an NVIDIA card
for this release, and nothing was built for an AMD card.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on
Linux or a Mac. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The strata-poly models are experimental, and on a Mac they are a beta.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12, AMD cards and Macs compile on your computer.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• GLM 5.3 Flash, DeepSeek V4 Flash, Kolibri 1 and Clover have not been tried on an AMD card.

Most of this project is the work of Strata's author and contributors.