TUFF 9.4.0

TUFF 9.4.0 makes long context windows free until a chat uses them, and fixes the benchmark for models bigger than your Mac's memory.

• Faster replies at large context sizes. TUFF used to set aside the memory for the whole context window as soon as a model loaded, for example 2.7 GB for Gemma 4 26B at 128K. On a 16 GB Mac that memory was missing from the file cache that streamed experts read from. TUFF now starts small and grows the cache as the conversation gets longer. On a 16 GB M2 Air at a 128K window: Gemma 4 26B went from 6.7 to 8.9 tok/s, Qwen 3.6 35B from 8.6 to 10.3, and GPT-OSS 20B (at 64K) from 6.1 to 7.5. Replies are word-for-word the same as before, including on prompts long enough to make the cache grow.
• The benchmark loads models the way chat does. It wasn't using Auto's plan for which weights stay in memory, so Qwen 3.8 27B tried to keep all 15.5 GB in memory and failed, and Ornith 397B ran with more memory than Auto allows and pushed the Mac into swap. Both now use the same settings as a new chat. On a 16 GB M2 Air the 27B finished the benchmark and passed its check, using 1.3 GB, and the 397B ran for 15 minutes without swapping, using about 0.5 GB. Both are still very slow on 16 GB (around 0.1 tok/s), because almost every token streams from the SSD.
• All engines are now 1.0.20 and download the next time you load a model.

The 9.2.0 benchmark also looked about 30% slower than 8.3.6. Running both versions back to back on the same Mac showed the same speed (6.6 tok/s each for Gemma 4 26B), so that gap came from conditions on the Mac during the run, not from TUFF.