TUFF 9.4.5

TUFF 9.4.5

A small follow-up to the move to the tuff-app organization.

• The web and file sandbox now runs sandbox image 2, with pypdf 6.21.2. It fixes this month's pypdf advisories, including memory blow-ups and infinite loops on crafted PDFs and unvalidated JPEG data (GHSA-5hx5-vpf9-5wcv, GHSA-6vvc-x485-93v7, GHSA-f54x-cmq8-qg57, GHSA-cgf9-327m-v77c). PDFs were already read in a separate process with time and memory limits, so these could at worst stop a PDF from being read. TUFF downloads the new image the next time the sandbox starts.
• The benchmark leaderboard link opens the website's new address, tuff-app.github.io/benchmarks.

TUFF is now split into a few repositories that share one set of policies: TUFF, sandbox, homebrew-tap and the website. The Homebrew cask moved too: new installs use brew install --cask tuff-app/tap/tuff. If you installed from the old tap, the homebrew-tap README has the three commands that move your install over.

Thanks to @Osiriss664 for the pypdf update (#57).

Engines are unchanged from 9.4.3. Your settings, chats and models stay where they are.

Validation: the full local source gate passed, including 2,682 Swift tests.

The Mac app is ad-hoc signed and not notarized. Model weights are separate downloads.

TUFF 9.4.4

TUFF 9.4.4

TUFF is moving to the new tuff-app organization on GitHub. This release only changes addresses, so nothing breaks when the repository moves:

• Updates, bug reports, benchmark sharing and the About panel now use github.com/tuff-app/TUFF.
• The website and benchmark leaderboard move to tuff-app.github.io/TUFF. Old links redirect.
• The web and file sandbox pulls its image from ghcr.io/tuff-app/tuff-sandbox. It's the same image with the same digest.
• Homebrew: the cask now lives in tuff-app/homebrew-tap, so new installs use brew install --cask tuff-app/tap/tuff. If you installed from the old tap, its README has the three commands that move your install over.

Your settings, chats and models stay where they are. Engines are unchanged from 9.4.3.

Validation: the full local source gate passed, including 1,139 Swift tests.

The Mac app is ad-hoc signed and not notarized. Model weights are separate downloads.

TUFF 9.4.3

TUFF 9.4.3

• Reject damaged saved-chat caches before restoring them. The cache checksum now covers its header and K/V data, and the runner validates every source offset before copying. Older caches are rebuilt automatically; saved chats and models are kept.
• Stop generation with an error when model arithmetic produces no valid token, instead of treating a failure as token zero or passing an invalid ID into the next forward pass.
• Keep low-temperature top-k sampling numerically stable. Very small positive temperatures could underflow every candidate weight and select the first token on every draw.
• Accept TUFF_LOG_CACHE for conversation diagnostics, while keeping TFF_LOG_CACHE working (#9).
• Keep the bug and performance forms' model choices consistent, including Heretic, and check for mismatches (#6).

Thanks to @arker2 for reporting the repeating-output problem in #53. A controlled test with Gemma 26B reproduced garbled output after corrupting a synthetic cache header and reloading. This patch addresses that confirmed restore defect and two numerical failure paths. The original intermittent report has not been reproduced with an intact cache, so #53 remains open for follow-up.

Validation: the full local source gate passed, including 2,682 Swift tests. The optimized Gemma engine also rejected the deliberately damaged synthetic cache and answered correctly after a cold prefill.

The Mac app is ad-hoc signed and not notarized. Model weights are separate downloads.

TUFF 9.4.2

TUFF 9.4.2 gets the inference engines ready before you load a model.

• Engines download in the background when TUFF opens. When an update ships new engines, the first load of each model family used to wait for its engine to download under "Preparing inference engine". TUFF now fetches the engines your installed models need as soon as it opens, so that wait is gone by the time you press Load. A load started while a download is still running just waits for that same download. Engines you already have are only checked.

I also went through the rest of the loading path looking for anything slower than 8.x and didn't find anything else. Setting up the engine's GPU code takes about 0.05 s, and checking a model's weights takes the same time it did before 9.0 (about 2.6 s for Ornith 1.5 9B on a 16 GB M2 Air).

TUFF 9.4.1

TUFF 9.4.1 fixes two bugs: the reload prompt that kept showing up after ordinary answers, and the Background API ignoring the context size set in Settings.

• No more "Settings changed. Reload the model to continue." after a simple answer (#54). With Auto on, TUFF re-planned the model's memory after every answer and changed the settings of the model that was already loaded, so it asked for a reload even though nothing had been changed. A memory warning, or any app on the Mac swapping a little during an answer, was enough to set it off. Auto still learns from every answer, but its new plan now waits for the next time you load the model. Paging under 64 MB no longer counts against a plan, and if every plan has been seen under pressure, Auto plans as if nothing had been observed instead of failing to load. Thanks to @arker2 for the report and @Osiriss664 for tracking down the cause.
• The Background API and tuff serve use each model's memory settings from Settings > Models (#55). The server always planned with Auto's Balanced profile, which capped most models at 16K. Now, with Auto on, it uses the profile you picked. With Auto off, it uses the Context you chose, as long as that fits your Mac's memory, and otherwise falls back to Balanced. /v1/models reports the new limit. On a 16 GB M2 Air with Ornith 1.5 9B set to 64K, a 31K-token prompt that used to be refused now gets an answer. Changes apply the next time the server loads the model. Thanks to @betamuh for the detailed report and patch.

TUFF 9.4.0

TUFF 9.4.0 makes long context windows free until a chat uses them, and fixes the benchmark for models bigger than your Mac's memory.

• Faster replies at large context sizes. TUFF used to set aside the memory for the whole context window as soon as a model loaded, for example 2.7 GB for Gemma 4 26B at 128K. On a 16 GB Mac that memory was missing from the file cache that streamed experts read from. TUFF now starts small and grows the cache as the conversation gets longer. On a 16 GB M2 Air at a 128K window: Gemma 4 26B went from 6.7 to 8.9 tok/s, Qwen 3.6 35B from 8.6 to 10.3, and GPT-OSS 20B (at 64K) from 6.1 to 7.5. Replies are word-for-word the same as before, including on prompts long enough to make the cache grow.
• The benchmark loads models the way chat does. It wasn't using Auto's plan for which weights stay in memory, so Qwen 3.8 27B tried to keep all 15.5 GB in memory and failed, and Ornith 397B ran with more memory than Auto allows and pushed the Mac into swap. Both now use the same settings as a new chat. On a 16 GB M2 Air the 27B finished the benchmark and passed its check, using 1.3 GB, and the 397B ran for 15 minutes without swapping, using about 0.5 GB. Both are still very slow on 16 GB (around 0.1 tok/s), because almost every token streams from the SSD.
• All engines are now 1.0.20 and download the next time you load a model.

The 9.2.0 benchmark also looked about 30% slower than 8.3.6. Running both versions back to back on the same Mac showed the same speed (6.6 tok/s each for Gemma 4 26B), so that gap came from conditions on the Mac during the run, not from TUFF.

TUFF 9.3.0

TUFF 9.3.0 can run web search, page reads and file search inside a sandbox.

A huge thanks to @Osiriss664, who designed and built all of this across four pull requests (#48, #50, #51 and #52, tracked in #38): the sandbox image and its server, the firewalls, the signed connection between TUFF and the sandbox, starting and stopping it, and the web and file search that go through it. That's over 20,000 lines of careful code and tests. Thank you!

• Settings > Search > Run web search and page reads in a sandbox. Searches and the pages the model reads are fetched inside a small Linux virtual machine, and pages and PDFs are turned into text there, so TUFF only receives finished text. The sandbox can't reach your Mac or your local network.
• Settings > Search > Files > Read my search folders in a sandbox too. A second sandbox with no network at all reads the folders you picked, read-only, and turns your files into passages for the search index.
• Both are off by default. They need macOS 26 or newer and Apple's container tool, version 1.5 or newer. The first start downloads the sandbox image (about 65 MB).
• If a sandbox isn't ready, web or file search fails with a message. It never falls back to reading pages or files in TUFF.
• File and folder names with hidden or direction-changing characters are now shown written out, so a name can't pretend to be another file type. This applies with or without the sandbox.

More details are in Web sandbox.

TUFF 9.2.0

TUFF 9.2.0 adds image support to all three Ornith 1.5 models.

• Ornith 1.5 9B, 35B-A3B and 397B can now read images. Install the Image Support add-on from the Models page. Each is about 900 MB, needs 16 GB of memory and an M2 or newer.
• The MLX conversions of Ornith 9B and 35B left out the vision part, so their add-ons come from Ornith's original releases of the same models. Both are pinned by revision and checksum, and a pack only installs for the exact model it was made for.
• On a 16 GB M2 Air, all three described a test image correctly. 9B answered in 42 s and 35B in 54 s. 397B took six and a half minutes for a one-word answer, so it's usable but slow.
• The Qwen engine and the others are now 1.0.19 and download the next time you load a model.

I also tried keeping guessed experts out of the cache until they're used. It removed all the cache misses caused by wrong guesses and cut expert reads by 10 to 20%, but decode speed only moved by -0.5% to +1.5%, so it isn't in this release. The details are in #24.

TUFF 9.1.2

TUFF 9.1.2 lets the command line and the local API learn like the app, and serves your custom checkpoints by name.

• tuff prompt and tuff serve now use the same Auto history as the app. Each completed run is recorded there too, so settings learned in one place carry over to the others. Records hold timings and settings only, never prompts or replies.
• Custom checkpoints you imported in Settings now show up in the local API under a name made from theirs. "My Tune" becomes my-tune, and you can request it like any other model. They're text only, as in the app.
• A missing model folder now says so, instead of reporting that no safe Auto allocation was found.
• All engines are now 1.0.18 and download the next time you load a model.

TUFF 9.1.1

TUFF 9.1.1 lets saved chats pick up where they left off.

Before, reopening a long chat after quitting TUFF or switching models meant reading the whole conversation again. Now TUFF saves the chat's state to disk and resumes from it.

On a 16 GB M2 Air, a follow-up in a 3,300-token chat after reloading the model:

Model	Before	Now	Saved state
Gemma 4 26B-A4B	114 s	6.8 s	334 MB
Qwen 3.6 35B-A3B	157 s	7.8 s	132 MB
GPT-OSS 20B	155 s	10.3 s	91 MB

• Only chats of 1,024 tokens or more are saved, and a chat is saved again only after it has grown by a quarter, so a long chat is written a few times, not after every reply.
• Saved state lives in TUFF's cache (2 GB limit, oldest removed first). It's deleted when you delete the chat or edit an earlier message.
• Set TUFF_SAVED_CHATS=off to keep chat state in memory only.
• All engines are now 1.0.17 and download the next time you load a model.