0.1.44.1-poly.3

LFM2.5-24B-A2B runs on PolyStrata, and strata-poly reads long prompts faster on a card that holds the model.

What changed since 0.1.44.1-poly.2

• LFM2.5-24B-A2B is the eighth model: ./setup.sh --family lfm25 installs one of five sizes on strata-poly, Unsloth's UD-Q4_K_XL, UD-Q3_K_XL, UD-Q6_K_XL and Q8_0 or Liquid AI's QAD-Q4_0 (11 to 25 GB). Most of its 40 layers have no attention, so a long conversation takes little VRAM: a 128K context takes 2 GB more than 32K. With UD-Q4_K_XL an RTX 4070 laptop (8 GB) writes a short chat at 70 to 73 tokens/s and reads a 30K-token prompt at 3,590 to 3,620 tokens/s; llama.cpp, tuned on the same laptop, read a 32K prompt at 530 and wrote after it at 53. An RTX 5090 reads the 30K prompt at 23,400 tokens/s, as fast as llama.cpp with a batch of 2,048, and after it writes one token at a time at 388 tokens/s where llama.cpp writes at 369. A 124K-token prompt in a 128K context was read and answered in 8.7 s on the 5090, and read in 45 s on the laptop. Windows or Linux, one NVIDIA card. Its license, the LFM Open License v1.0, allows commercial use only to companies with less than 10 million dollars of yearly revenue. More in docs/LFM.md.
• Faster prompt reading on a card that holds every expert: a prompt batch runs its cached experts in three launches a layer instead of nine. Qwen3.6 UD-Q2_K_XL on an RTX 5090 reads a 30K prompt in 3.7 s instead of 3.9 to 4.0, a 2K prompt in 333 to 338 ms instead of 371 to 376. Setup now reads prompts 2,048 tokens at a time where the card holds every expert of LFM2.5 or Qwen3.6 beside the larger batch, and for LFM2.5 on any card from 8 GB. The products with few outputs (routers, a gate a head) read their next inputs while they work, with the same bits: Qwen3.6's took 82 ms of a 7.7K read instead of 126. GLM 5.3 Flash, which copies half its experts as it reads, reads as fast as before.
• Faster writing on a card that holds every expert: a few tokens' experts are routed on the card, and a token's layers run as one recorded launch. On the 5090, one token at a time, Qwen3.6 UD-Q2_K_XL writes at 294 tokens/s instead of 279 and Gemma 4 UD-Q2_K_XL at 254 instead of 245.
• The server reads LFM2.5's tool calls, a Python list between two tokens, for the OpenAI and Anthropic APIs, and its chat template.

The engine changed, so the zip below is a new ready-made Windows engine; the installer downloads it and checks its SHA-256.

How fast

The README has the tables, now with LFM2.5-24B-A2B ahead of Qwen3.6, and Qwen3.6's, Ornith's and Gemma 4's workstation numbers measured again; docs/LFM.md and docs/BACKENDS.md have the runs.

Install

Download the source below, or git clone the repository. Then run START-HERE.bat on Windows or ./setup.sh on Linux. An earlier install updates with UPDATE.bat or ./update.sh.

Limits

• The seven added models are experimental. LFM2.5 has run on an 8 GB laptop and a 32 GB card, and not on an AMD card.
• The ready-made engine is for Windows and CUDA 13 only; CUDA 12 and AMD cards compile on your PC.
• GLM 5.3 Flash and DeepSeek V4 Flash need 70 to 100 GB of RAM, and about 8 GB more than their files in a container.
• AMD cards on Windows run Qwen3.8-Flash-Next only.

Most of this project is the work of Strata's author and contributors.