Skip to content

sushi v1.7.0-dev4

Pre-release
Pre-release

Choose a tag to compare

@beamivalice beamivalice released this 17 Nov 07:57

Development build of main (fb34fe6). It is not the latest release: sushi update, the built-in update check and Homebrew stay on v1.6.0 and never offer it. Ad-hoc signed and not notarized, like every sushi release.

To try it:

shasum -a 256 -c sushi-bin-macos-arm64.tar.gz.sha256
tar -xzf sushi-bin-macos-arm64.tar.gz
xattr -dr com.apple.quarantine sushi-macos-arm64
./sushi-macos-arm64/sushi --version

Since v1.7.0-dev3:

  • Qwen3.8 runs on Sushi Cooler, the engine of our own, by default: its STQ packs and its BF16 checkpoint. Pass --qwen-mlx-engine, or clear the web UI's qwen_cooler checkbox, to keep it on MLX; the load line names the side.
  • A Qwen3.8 load on Sushi Cooler runs its vision tower there too, and decodes concurrent requests as joined rows, as on MLX.
  • A Sushi Cooler load compiles nothing inside a request: its warm-up makes what a request can pick, and what a request still had to compile is kept for the next load.
  • Sushi Cooler holds less memory per prefill chunk, and a prompt of several chunks reserves its KV cache once.
  • DeepSeek-V4.1 on Sushi Cooler at --ssd-budget-gb 108 warms within the memory it has and counts its parked decode pool once.
  • Quality on Sushi Cooler (KLD to the first EOS, release-gate fixtures): Qwen STQ 2.75bpw 0.061793 (v1.6.0 on MLX 0.061834); DeepSeek STQ 1.75bpw 0.210682 (v1.6.0 0.210305).

Known in this build:

  • A 4096-token Qwen3.8 prefill chunk on Sushi Cooler can hold about 19% more memory than it was admitted under.
  • Builds of a single architecture (-Darchs) without DeepSeek-V4.1 or without Qwen3.8 fail some tests; the release binary carries every architecture and is not affected.