ishizuki v0.1.4

Native macOS inference engine for Ternary Bonsai 2 27B on MLX Swift — 2-bit
Hadamard-rotated weights, vision, tool calling, OpenAI + Anthropic APIs. No Python.

Requires Apple Silicon and macOS 15+.

Install

Installer — ishizuki-v0.1.4.pkg, signed and notarized, installs to ~/.local, no admin
password. Uninstall with ~/.local/libexec/ishizuki/uninstall.sh.

Loose binary — ishizuki-v0.1.4-macos-arm64.tar.gz: the executable and the metallib it
loads. Put the two anywhere, together.

tar -xzf ishizuki-v0.1.4-macos-arm64.tar.gz
./ishizuki-v0.1.4-macos-arm64/ishizuki serve

Since v0.1.3

• Prefix caches survive the idle timer. --idle-timeout was dropping every cached prefix
  after 120s, so a pause between turns cost a full re-prefill — tens of thousands of tokens
  discarded to reclaim a buffer pool the residency timer had already freed on its own. Idle now
  releases only that pool; prefix caches are dropped when the model is unloaded
  (--evict-timeout).
• The budget carves out a full context first. Weights and one full context of KV are
  reserved before the buffer pool gets anything, and prefix slots scale with what is left.
  serve prints the reserve and warns when the wired ceiling cannot hold weights plus one full
  context.

Defaults on a 64 GB machine, 3.5-bit KV:

budget     ceiling 54.0 GB, context reserve 4.0 GB, buffer cache 8.0 GB, 6 prefix slots

fp16 KV reserves 16.5 GB and drops to 2 slots; --context-scale 2 reserves 8.0 GB.

Known issue

A std::out_of_range inside MLX's graph eval (Fatal error: vector) has been reported during a
long generation with thinking enabled. It is not reproduced here yet and is not fixed.