Skip to content

TUFF 9.2.0

Choose a tag to compare

@rexmhall09 rexmhall09 released this 14 Oct 10:01
· 22 commits to main since this release

TUFF 9.2.0 adds image support to all three Ornith 1.5 models.

  • Ornith 1.5 9B, 35B-A3B and 397B can now read images. Install the Image Support add-on from the Models page. Each is about 900 MB, needs 16 GB of memory and an M2 or newer.
  • The MLX conversions of Ornith 9B and 35B left out the vision part, so their add-ons come from Ornith's original releases of the same models. Both are pinned by revision and checksum, and a pack only installs for the exact model it was made for.
  • On a 16 GB M2 Air, all three described a test image correctly. 9B answered in 42 s and 35B in 54 s. 397B took six and a half minutes for a one-word answer, so it's usable but slow.
  • The Qwen engine and the others are now 1.0.19 and download the next time you load a model.

I also tried keeping guessed experts out of the cache until they're used. It removed all the cache misses caused by wrong guesses and cut expert reads by 10 to 20%, but decode speed only moved by -0.5% to +1.5%, so it isn't in this release. The details are in #24.