Tempo9 v1.2.1

Tempo9 1.2.1

Engine update only; the CLI and the Swift packages are unchanged apart from the engine they link.

Long-context prefill on Gemma 4 (and every hd256 / hd512 attention model)

The paged NAX attention kernel now splits the head dimension across simdgroups. Gemma 4's global attention layers (head_dim 512, one KV head) previously held a 32-tile accumulator per simdgroup and ran at about 3 TFLOPS, which was 90% of a 96k-token prefill. Measured on one M5 Pro 24 GB, same GGUF, same prompt (85% of the context, 32-token answer):

model	prompt	1.2.0	1.2.1	llama.cpp 0.7.0
Gemma 4 E2B	24k tokens	7.1 s	4.6 s	10.6 s
Gemma 4 E2B	48k tokens	22.8 s	13.4 s	28.3 s
Gemma 4 E2B	96k tokens	77.8 s	43.4 s	77.7 s
Gemma 4 E4B	96k tokens	105 s	68 s	100 s
Qwen3.5-9B	28k tokens	32 s	28.5 s

Greedy output is byte-identical to 1.2.0 on Gemma 4 E2B, E4B and Qwen3.5-9B. Off-switches for A/B: AS_METAL_FA_MLX_HD512_DS=1, AS_METAL_FA_MLX_HD256_DS=1.

Full write-up: https://www.thinkspread.com/blog/gemma4-e-tempo9-vs-llamacpp/

Tempo9 v1.2.0

New engine, and Flint runs.

Flint

tempo9 --gguf Flint-*.gguf works. The Swift side got Flint's staged pre-tokenizer and end-of-turn tokens; the engine got attention sinks, sliding windows, NoPE layers and sigmoid routing on the Qwen3-MoE graph. Our Q6K-dense build is on the Hub as thinkspread/Flint-Q6K-IQ3XXS-GGUF (18.0 GB). Its top-1 agreement with the Q8 reference is 85.3 %, the official int4's is 85.1 %, and it is 31 % smaller. With 16 clients on an M5 Pro it serves 2.4× the aggregate tokens of Recursive's own engine, which runs one session at a time.

Decode is a bit faster on every MoE model (Metal)

• The routing GEMV now reads a bf16 copy of the fp32 gate weights (AS_METAL_ROUTER_MV=2, default; =1 puts fp32 back). Storage stays fp32. Flint +1.5–2.8 %, Qwen3.5-35B +1–3 %; greedy output byte-identical on both.
• Greedy argmax is split across threadgroups instead of one threadgroup walking the whole vocabulary: −0.15 ms per token at 150k vocab.
• The KV append rides inside the qknorm+rope dispatch: −0.13 ms per token.
• Decode RMSNorm launches 1024 wide, same bits: −0.04 ms per token.
• Decode batches of 9–32 requests used to fall onto the prefill MoE path. Fixed. Flint c16 192 → 378 tok/s, Qwen3.5-35B c16 166 → 272.

Long prompts

Attention sinks reach the MLX prefill kernel now. Flint, 8K-token prompt: 5.30 → 3.74 s.

Idle

The three engine loops back off to a 2 ms sleep after 100 ms of nothing to do. Idle server: 7.2 % → 1.0 % of a core. Throughput and mid-decode arrivals unchanged. AS_ENGINE_IDLE_SLEEP_US=200 turns it off.

API

• Anthropic usage has cache_read_input_tokens / cache_creation_input_tokens; cache-point boundaries are remembered across turns.
• A tool call's name is mapped onto the tool the client actually declared.

Engine: dash-infer 754721a8f (C ABI 3.0). All numbers from one M5 Pro 24 GB, macOS 26, 2026-10-23..26. The long version is the post Flint Inference on Apple Silicon: rec Megakernel and Tempo9 Compared.

Tempo9 v1.1.3

One command from an empty Mac to a running server:

brew install thinkspread/tap/tempo9
tempo9 --hf unsloth/Qwen3.5-9B-GGUF:Q4_K_S

--hf owner/repo[:quant] downloads a GGUF into the standard Hugging Face cache, so a copy that huggingface_hub or llama.cpp already pulled is reused. Downloads resume after an interruption and are checked against their sha256. With no quant given, it picks Q4_K_S.

Claude Code reuses its prompt cache.

• The server forwards Anthropic cache_control breakpoints to the engine, and pins the points in the tool list where Claude Code's prompt changes between turns.
• The new engine keeps recurrent-state snapshots for hybrid models such as Qwen3.5, so a request whose tool list changed can resume from cache instead of starting over.
• On Qwen3.5-9B (M5 Pro), a one-question Claude Code session goes from about 98 s to about 63 s, and a later session on the same server takes 10–23 s.
• manual/claude-code.md breaks down what a session costs and compares Qwen3.5-9B with Gemma-4-E4B.

Docs. The README now carries the 2026-10 benchmark against vllm-metal, omlx and llama.cpp; the full write-up is docs/benchmark-2026-10-m5pro.md.

The CLI is signed and notarized (Developer ID).

Tempo9 v1.1.2

Engine 3.0.0-rc3 (C ABI 3.0).

• Metal tensor units are back on. They had been off in every build since 2026-09-23, including 1.1.1. Prefill is faster on M5-class GPUs. q8_0 and RMSNorm prefill staging is cheaper, and q8_0 decode at batch 9–16 uses a small-M tensor tile.
• Gemma 4 E-series: prefill computes the 18 KV-shared layers on the last token only, and Gemma 4 now gets the prefix cache.
• Tool calls: a past tool call's arguments string is decoded before the chat template renders it. Gemma 4 tool calls no longer leak into the message content, and generation stops at <eos>.
• Tokenizer: special tokens are split in one byte-level pass, about 15× faster on agent prompts.

Install: brew install thinkspread/tap/tempo9, curl -fsSL https://raw.githubusercontent.com/thinkspread/tempo9/main/install.sh | bash, or SwiftPM .package(url: "https://github.com/thinkspread/tempo9", from: "1.1.2").

The CLI is signed and notarized (Developer ID).

Tempo9 v1.1.1

Tempo9 1.1.1 is a bug-fix release. It ships a new engine with two correctness fixes; if you use 1.0.0 or 1.1.0, upgrade.

Requirements: unchanged. You need a Mac with Apple Silicon and macOS 26 or later. GPU (Metal) acceleration currently needs an M5 or later; earlier chips run on the CPU.

Fixes

• BF16 / F16 models produced garbage. Loading a model whose weights are all BF16 or F16 (for example Qwen3-0.6B-BF16.gguf) assembled every fused QKV weight wrong: 2-byte elements were copied into a buffer laid out for 4-byte ones. Output was unreadable for full-attention models and never stopped for hybrid ones. Quantized models (Q4_K, Q8_0, …) were not affected.
• About 3–4% of responses were garbled under concurrent load on Metal. When several requests shared a long prompt prefix and decoded together, a one-step-lag decode pipeline let a host/GPU race corrupt some outputs. The pipeline is now off by default: 0 bad answers in 400 shared-prefix requests at 8 in flight, against 14–15 with it on. Single-stream decode is 1.3–2.1% slower; concurrent decode is not slower.
• The server no longer turns text into tool calls when the request declared no tools. A model writing <tool_call> in a plain chat request used to get an empty reply with finish_reason: "tool_calls". Now the text is returned as text; requests that do declare tools behave as before.

The engine's C header gains te9_gemm_backend_reason() (engine API 1.5), which says why the engine fell back to the CPU. It is additive; nothing in the Swift API changed.

Install or upgrade the tempo9 server

brew upgrade tempo9        # or: brew install thinkspread/tap/tempo9

or

curl -fsSL https://raw.githubusercontent.com/thinkspread/tempo9/main/install.sh | bash

Use the engine in-process (Swift)

.package(url: "https://github.com/thinkspread/tempo9", from: "1.1.1")

A package already on from: "1.1.0" picks up 1.1.1 with swift package update. Package.swift now points at this release's Tempo9Engine.xcframework.zip.

Licence

The licences have not changed. The engine is under the Tempo9 Engine License 0.1 (ENGINE-LICENSE), and the source in this repository is under Apache-2.0.

Assets

File	SHA-256 / SwiftPM checksum
tempo9-1.1.1-macos-arm64.tar.gz	35def0f05f7ea4c9710857aa81b888bfd1b767d63f782200dafdfcd2ef855cca
Tempo9Engine.xcframework.zip	c7a654c031b91792009eea4ecdda6ce078fd2a5f6c7a3fc1eaf66b7760d2a407

Tempo9 v1.1.0

Tempo9 1.1.0 makes two more of the SDK's modules public, for apps that embed the engine.

Requirements: unchanged. You need a Mac with Apple Silicon and macOS 26 or later. GPU (Metal) acceleration currently needs an M5 or later; earlier chips run on the CPU.

What's new

• VisionTowerKit is a library product. LocalSession takes ImageEmbeddings, and VisionTowerKit produces them. Before this release, an app outside the package had no way to show the model an image.
• GGUFKit is a library product. It reads a .gguf file (metadata, tensors, tokenizers) using Foundation alone.
• Tempo9.EngineInfo reports the engine build (version) and the GEMM backend actually running (gemmBackend: "metal" or "cpu").
• New tests cover ToolCallParser in both dialects it reads (Qwen XML and Gemma 4), and the Gemma 4 channel tags in the reasoning splitter.

The engine has not changed. Package.swift still points at the 1.0.0 Tempo9Engine.xcframework, and the server behaves exactly as in 1.0.0. The tempo9 binary was rebuilt against that same engine, then signed and notarized as 1.1.0.

Install or upgrade the tempo9 server

brew upgrade tempo9        # or: brew install thinkspread/tap/tempo9

or

curl -fsSL https://raw.githubusercontent.com/thinkspread/tempo9/main/install.sh | bash

Use the engine in-process (Swift)

.package(url: "https://github.com/thinkspread/tempo9", from: "1.1.0")

Then add the products you use:

.product(name: "Tempo9", package: "tempo9"),
.product(name: "VisionTowerKit", package: "tempo9"),  // image or audio input
.product(name: "GGUFKit", package: "tempo9"),         // reading model files yourself

Declare platforms: [.macOS("26.0")] in your package. See docs/swift-sdk.md.

Licence

The licences have not changed. The engine is under the Tempo9 Engine License 0.1 (ENGINE-LICENSE), and the source in this repository is under Apache-2.0.

Assets

File	SHA-256
tempo9-1.1.0-macos-arm64.tar.gz	d6bfed23a4feb526360216dcf0518e8e9ed2fc69f05e1dc8a5937aafb199d168

The engine is the Tempo9Engine.xcframework.zip attached to v1.0.0 (SwiftPM checksum 6d8e98932499ab4494fa29e2d4a529281bae5428eb25c6eccdb68f4e30e6eb1a).

Tempo9 1.0.0

The first public release of Tempo9: server-grade LLM inference on Apple Silicon.

Requirements: a Mac with Apple Silicon and macOS 26 or later. GPU (Metal) acceleration currently needs an M5 or later; earlier chips run on the CPU.

Install the tempo9 server

brew install thinkspread/tap/tempo9

or

curl -fsSL https://raw.githubusercontent.com/thinkspread/tempo9/main/install.sh | bash

The binary is signed with a Developer ID and notarized by Apple.

Use the engine in-process (Swift)

.package(url: "https://github.com/thinkspread/tempo9", from: "1.0.0")

SwiftPM downloads Tempo9Engine.xcframework (attached below). Declare platforms: [.macOS("26.0")] in your package; see docs/swift-sdk.md.

Licence

The engine is licensed under the Tempo9 Engine License 0.1 (ENGINE-LICENSE). It is free for individuals and for organisations with fewer than 100 employees and under US$1M in annual revenue; above that, a commercial licence is required. The source in this repository is Apache-2.0. NOTICE lists the open-source components inside the engine.

Assets

File	SHA-256
tempo9-1.0.0-macos-arm64.tar.gz	660a759c04f6f8958940548eae5b4cc8e33955e0c4b8fa4a2b769b247aa3e437
Tempo9Engine.xcframework.zip	6d8e98932499ab4494fa29e2d4a529281bae5428eb25c6eccdb68f4e30e6eb1a (SwiftPM checksum 6d8e98932499ab4494fa29e2d4a529281bae5428eb25c6eccdb68f4e30e6eb1a)