Skip to content

sushi v1.0.4

Choose a tag to compare

@github-actions github-actions released this 26 Sep 11:37
· 703 commits to main since this release

v1.0.4 — Faster prompts and browser chat

  • Faster prompts on M1–M4: Macs without the M5's neural accelerators read prompts about 4x faster on the EXL3
    packs (M2 Max, 3–4k-token prompts, default settings).
  • Faster Flash-Next prompts on 64 GB Macs: when the n-gram table cannot stay in memory beside the model, prompt
    processing reads it in parallel by default instead of one row at a time.
  • Smaller contexts need less free memory to load: resident Flash-Next EXL3 packs without separate sidecars or
    ANE now size load headroom from the chosen context instead of always asking for 7 GB above the weights.
  • Chat in your browser: sushi serve and sushi run serve a chat page at http://127.0.0.1:12345/ that streams
    replies, shows the model's thinking, takes images for vision models and keeps your conversations in the browser.
  • /cd <folder> in sushi run: moves the folder the file tools and relative /image paths read from, the prompt
    always shows that folder and whether tools are on, and a model that asks for a file outside it now suggests /cd.