0.1.44.1-poly.5

MiniCPM-V 4.7 35B-A3B reads pictures on PolyStrata, the Qwen3.6 family can keep its attention cache in 8 bits, and a model can keep the thinking of earlier turns.

What changed since 0.1.44.1-poly.4

• MiniCPM-V 4.7 35B-A3B by OpenBMB is the ninth model, brought in by iMilnb (PR 20): ./setup.sh --family minicpmv installs one of three sizes, Q4_K_M, mix_q3gud_im or Q6_K (16 to 29 GB). It has Qwen3.6-35B-A3B's architecture and its own weights, made to read pictures. A large picture is read as an overview and a few tiles, and on four test pictures, from a 64x64 icon to a 2520x1620 chart, it read the same numbers and labels as llama.cpp's own llama-mtmd-cli. Its file has no draft block, so it writes one token at a time: with Q4_K_M an RTX 4070 laptop (8 GB) writes a short chat at 64 tokens/s and reads a 32K-token prompt at 1,150; an RTX 5090 writes at 308 to 311 and reads at 7,910 to 7,950. Its license is Apache 2.0.
• --kv q8_0 for Qwen3.6, Ornith, Tiel Coder and MiniCPM-V, by vpkoskinen (PR 23): the attention cache in ggml's Q8_0 blocks, about half its VRAM, which the expert cache can use at long contexts. It is off unless you add it to the run config's args; without it the engine gives the same text as before. On the 5090 it wrote 2% to 6% slower after an 8K prompt; on the 8 GB laptop there was no difference to measure.
• "preserve_thinking": true in strata-<model>.json, or "chat_template_kwargs": {"preserve_thinking": true} in a request (issue 25): Qwen3.6's template, which Ornith and Tiel Coder share, then keeps the thinking of earlier turns in the prompt instead of dropping it, and the server remembers the thinking of the answers it wrote, so it works with apps that send an answer back without it (the chat page, Open WebUI, VS Code's Copilot). In a test where the model thinks of two 20-digit numbers and says one, with the app dropping the thinking, it gave the other exactly when asked later, and made one up without the setting.
• The picture encoder builds from llama.cpp 69f201a, which MiniCPM-V needs; the two engines stay on the Strata project's 3cf03257.

The engine and the picture encoder changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

How fast

The README has MiniCPM-V's tables for both PCs, next to the other models'.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh, and START-HERE.bat --setup --family minicpmv adds MiniCPM-V.

Limits

• The eight added models are experimental.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• --kv q8_0 is for the qwen35moe models only; the others refuse it at start. It has not been tried on an AMD card, where the engine refuses it.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.