Tempo9 v1.2.0

New engine, and Flint runs.

Flint

tempo9 --gguf Flint-*.gguf works. The Swift side got Flint's staged pre-tokenizer and end-of-turn tokens; the engine got attention sinks, sliding windows, NoPE layers and sigmoid routing on the Qwen3-MoE graph. Our Q6K-dense build is on the Hub as thinkspread/Flint-Q6K-IQ3XXS-GGUF (18.0 GB). Its top-1 agreement with the Q8 reference is 85.3 %, the official int4's is 85.1 %, and it is 31 % smaller. With 16 clients on an M5 Pro it serves 2.4× the aggregate tokens of Recursive's own engine, which runs one session at a time.

Decode is a bit faster on every MoE model (Metal)

• The routing GEMV now reads a bf16 copy of the fp32 gate weights (AS_METAL_ROUTER_MV=2, default; =1 puts fp32 back). Storage stays fp32. Flint +1.5–2.8 %, Qwen3.5-35B +1–3 %; greedy output byte-identical on both.
• Greedy argmax is split across threadgroups instead of one threadgroup walking the whole vocabulary: −0.15 ms per token at 150k vocab.
• The KV append rides inside the qknorm+rope dispatch: −0.13 ms per token.
• Decode RMSNorm launches 1024 wide, same bits: −0.04 ms per token.
• Decode batches of 9–32 requests used to fall onto the prefill MoE path. Fixed. Flint c16 192 → 378 tok/s, Qwen3.5-35B c16 166 → 272.

Long prompts

Attention sinks reach the MLX prefill kernel now. Flint, 8K-token prompt: 5.30 → 3.74 s.

Idle

The three engine loops back off to a 2 ms sleep after 100 ms of nothing to do. Idle server: 7.2 % → 1.0 % of a core. Throughput and mid-decode arrivals unchanged. AS_ENGINE_IDLE_SLEEP_US=200 turns it off.

API

• Anthropic usage has cache_read_input_tokens / cache_creation_input_tokens; cache-point boundaries are remembered across turns.
• A tool call's name is mapped onto the tool the client actually declared.

Engine: dash-infer 754721a8f (C ABI 3.0). All numbers from one M5 Pro 24 GB, macOS 26, 2026-10-23..26. The long version is the post Flint Inference on Apple Silicon: rec Megakernel and Tempo9 Compared.