nuclis v0.7.0

nuclis serve now speaks OpenAI's Chat Completions for the language models, streamed, with tools and images, and reuses what it has already read: on Qwen3.8 the third request of a conversation found 65 of its 86 prompt tokens cached. Decisions no longer wait behind a generation, answered in 0.05–0.16 s during a 2,000-token reply instead of after 136 s, and every open model now shares one memory budget. A new kind of model, EmbeddingGemma 2, turns text, images and audio into vectors through nuclis embed and an OpenAI-compatible POST /v1/embeddings, its BF16 file matching Google's own vectors to within 1e-10 of cosine, and the default file embedding 38 inputs of 256 tokens a second on an M4 Pro. The server's default port is now 9000 (it was 8000).

What's Changed

• build(release): make release-draft, release, and tag by @tildaslashalef in #5
• docs: retire the worklog and the unit identifiers by @tildaslashalef in #6
• refactor(serve): a shared completer and a streaming transport by @tildaslashalef in #7
• feat(serve): OpenAI Chat Completions for the language models by @tildaslashalef in #8
• feat(serve): streamed chat completions, keepalives, and cancellation by @tildaslashalef in #9
• feat(serve): decisions beside a generation, within one memory budget by @tildaslashalef in #10
• docs(spec): the Responses API is not planned by @tildaslashalef in #11
• feat(serve): structured fields for language models in GET /v1/models by @tildaslashalef in #12
• feat(serve): reuse model state across Gemma 4 turns and under a shared system prompt by @tildaslashalef in #13
• fix(serve): a fresh prefill clears the image flag, so later text conversations keep their states by @tildaslashalef in #14
• feat!: EmbeddingGemma 2, text, image and audio embeddings by @tildaslashalef in #15
• chore(release): v0.7.0 by @tildaslashalef in #16

Full Changelog: v0.6.0...v0.7.0

---

The attached build

macOS on Apple Silicon, release mode, with the Metal backend. It is not
signed or notarized — there is no Apple Developer ID for this project — so
macOS quarantines it on download. After unpacking:

shasum -a 256 -c SHA256SUMS               # optional, and worth it
gh attestation verify nuclis-*-aarch64-macos.tar.gz -R tildaslashalef/nuclis  # built here
tar -xzf nuclis-*-aarch64-macos.tar.gz
xattr -d com.apple.quarantine nuclis-*/nuclis
./nuclis-*/nuclis --help

A source archive (nuclis-*-src.tar.gz) is attached alongside the binary.
Building from source needs only Zig and the Command Line Tools, and takes
about a minute:

zig build -Doptimize=ReleaseSafe

Every number in this repository was measured on one machine — an M4 Pro with
48 GiB — and nothing has been run on other hardware. Full methodology is in
docs/benchmarks/README.md.

nuclis v0.6.0

v0.6.0 re-measures every model on macOS 27 against llama.cpp at four prompt lengths: Qwen3.8 decodes faster than the reference at every length (1.04–1.15×), Muse Glimmer's decode is up 9–22 %, and Gemma 4 E4B gets its first record.

nuclis now has a website, https://nuclis.dev, with the numbers on it. The documentation is reorganised into guides, models, the engine, the app, and benchmarks, every link checked by make docs-check.

Development moves to pull requests, each one the record of its work, and releases to signed tags whose notes list them. The worklog of the units before this release is archived at this tag.

What's Changed

• build(release): pull requests as the record, release-please for releases by @tildaslashalef in #1
• build(release): releases are an annotated tag; release-please removed by @tildaslashalef in #3
• chore(release): v0.6.0 by @tildaslashalef in #4

New Contributors

• @tildaslashalef made their first contribution in #1

Full Changelog: v0.5.0...v0.6.0

---

The attached build

macOS on Apple Silicon, release mode, with the Metal backend. It is not
signed or notarized — there is no Apple Developer ID for this project — so
macOS quarantines it on download. After unpacking:

shasum -a 256 -c SHA256SUMS               # optional, and worth it
gh attestation verify nuclis-*-aarch64-macos.tar.gz -R tildaslashalef/nuclis  # built here
tar -xzf nuclis-*-aarch64-macos.tar.gz
xattr -d com.apple.quarantine nuclis-*/nuclis
./nuclis-*/nuclis --help

A source archive (nuclis-*-src.tar.gz) is attached alongside the binary.
Building from source needs only Zig and the Command Line Tools, and takes
about a minute:

zig build -Doptimize=ReleaseSafe

Every number in this repository was measured on one machine — an M4 Pro with
48 GiB — and nothing has been run on other hardware. Full methodology is in
docs/benchmarks/README.md.

nuclis v0.5.0

The agent gets faster per step. A token cache keeps the primed prompt and each turn's end in memory and on disk, so a session resumes without prefilling it again (a 9.8K-token resume 126.5 → 2.05 s; task wall time −23 %). Tools read less (pages, outlines, grouped grep, short diffs; tool-result tokens −28 %), and a new command surface with /model switches models in place. Breaking: the slash commands changed.

Agent

• AGNT-19 Token caching for the agent: turn-boundary snapshots in memory and on disk; session management
• AGNT-20 A faster agent step: the command surface and /model, reading less, and the fixes from testing

Repository

• REPO-31 CI installs Zig 0.17.0

Breaking changes

• agent: the command surface and /model

All 19 commits

• 6cc9ec8 chore(release): v0.5.0
• 320bfeb docs: close AGNT-20, the command surface, reading less, and the fixes from testing
• ba75456 feat(config): max_tokens 8192 and a reasoning cap at every effort (AGNT-20)
• 6fe11f1 feat(agent): read less: pages, outlines, grouped grep, short diffs, repeat pointers (AGNT-20)
• 4c55dfc fix(inference): bound reopened reasoning, drop cut calls and stray closes, name bad tool arguments (AGNT-20)
• c141cd5 fix(agent): Enter on the command list runs the highlighted command; a bare slash is not a prompt (AGNT-20)
• b86c5d1 test(agent): measure tool-result tokens per tool, add the about task (AGNT-20)
• 1007b08 feat(agent)!: the command surface and /model (AGNT-20)
• 6bfae8c docs: close AGNT-19, the agent's token cache and session management
• e8a26b3 docs: plan AGNT-20, the command surface and /model, reading less, faster prefill
• c1bb45a fix(agent): print mode's --resume restores the last turn from the token cache (AGNT-19)
• db0ae43 fix(agent): a closed picker leaves the bar ready (AGNT-19)
• 4e10876 feat(agent): resume from the token cache, /list, /delete, agent rm, nuclis cache, Esc cancels (AGNT-19)
• 4df6f9f fix(agent): bash spawns from the process directory, fuzzy @ completion (AGNT-19)
• 3b43873 feat(agent): token cache for the primed prefix and turn ends, in memory and on disk (AGNT-19 session 1)
• 9f45bf0 docs: AGNT-19 adds /list, /delete, and agent rm to session 2
• 7bc3a3a fix(ci): release installs Zig from the tag's version and the workflow's digests (REPO-31)
• 900e5ea fix(ci): pin Zig 0.17.0 tarball digests for install-zig (REPO-31)
• f3af8ee chore: begin 0.5.0-dev

Full diff: v0.4.0...v0.5.0

---

The attached build

macOS on Apple Silicon, release mode, with the Metal backend. It is not
signed or notarized — there is no Apple Developer ID for this project — so
macOS quarantines it on download. After unpacking:

shasum -a 256 -c SHA256SUMS               # optional, and worth it
gh attestation verify nuclis-*-aarch64-macos.tar.gz -R tildaslashalef/nuclis  # built here
tar -xzf nuclis-*-aarch64-macos.tar.gz
xattr -d com.apple.quarantine nuclis-*/nuclis
./nuclis-*/nuclis --help

A source archive (nuclis-*-src.tar.gz) is attached alongside the binary.
Building from source needs only Zig and the Command Line Tools, and takes
about a minute:

zig build -Doptimize=ReleaseSafe

Every number in this repository was measured on one machine — an M4 Pro with
48 GiB — and nothing has been run on other hardware. Full methodology is in
docs/reference/bench.md.

nuclis v0.4.0

nuclis moves to Zig 0.17.0 with Qwen's speed unchanged (10.0 tok/s at 4K under both compilers). Speculation is re-priced per family and on by default where it pays: Qwen at draft 7 writes short code at 20.3–20.9 tok/s, helped by the DeltaNet replay tape and cheaper verify attention, and plain decode rises from 10.37 to 11.78 tok/s. Two decision models, Laya (ModernBERT) and Cloudflare's clef-flash, run on both backends and are served over HTTP by the new nuclis serve. Also: safetensors downloads, shell completion, and make verify in a third of the time (704 → 246 s).

Models

• MODL-28 Safetensors sets through the Hub client and model pull
• MODL-29 A generic safetensors loader and nuclis inspect on it
• MODL-30 Laya on the CPU end to end: the oracle, tokenizer.json, ModernBERT and the decision head, nuclis decide
• MODL-31 Laya on Metal: packed batches, bidirectional windowed attention over sequence bounds
• MODL-32 The Hub listing keeps its digests in ReleaseFast builds
• MODL-33 Laya multilingual: the Metaspace tokenizer, the checkpoint's own special tokens, checked on both backends
• MODL-34 clef-flash: Cloudflare's 9B decision model, text and vision, on both backends

Engine

• ENGN-18 The speed loop: saved prefixes, verify cost at depth, interleaved A/B, and the decode-speed baseline
• ENGN-19 The DeltaNet replay tape
• ENGN-20 Speculation re-priced per family; the catalogue's defaults

Kernels

• KERN-19 A threaded CPU reference, bit-identical: the CPU tier in 22 minutes
• KERN-20 Seeing inside the GPU: Metal captures, pipeline statistics, apple-gpu.md
• KERN-21 Few-query verify attention through the split pass
• KERN-22 Long-context decode attention: dropped before it started
• KERN-23 Weight streaming for one row and a few: Qwen decode +14 %, 2–3-row verify batches 6–12 % cheaper
• KERN-24 The register-fragment verify matmul

Agent

• AGNT-18 The agent's decide tool: dropped before it started
• AGNT-19 Token caching for the agent: snapshots at turn boundaries, in memory and on disk (in progress)

Application

• APPS-18 Shell completion answered by the binary; make install
• APPS-19 nuclis serve, the nuclis API: decisions first, batched across requests
• APPS-20 nuclis serve configured and logged: the serve section, port 8000, the default model at start, a line per request

Terminal

• TERM-13 Exit keeps the transcript

Repository

• REPO-18 docs/research/ removed
• REPO-19 The Gemma 4 12B K-quant file's gates retired with the file
• REPO-20 Fast verification: gates re-derived from code paths, a release tier, make verify-auto
• REPO-21 A generated playground; the Python scripts typed and formatted
• REPO-22 .reference/ for durable local state; .zig-cache disposable
• REPO-23 The README's Laya section: the experiment and its results
• REPO-24 Qwen decode at 20 tokens/s: evidence and experiment proposal
• REPO-25 A gate whose model is absent is skipped, not failed
• REPO-26 The README's speculative decoding results
• REPO-27 External review fixes: the Metal discard path, the kernel table, the downloader, parser property tests
• REPO-28 The README presents clef-flash beside Laya
• REPO-29 Zig 0.17.0: the tree migrated and its features adopted
• REPO-30 Zig 0.17 holds Qwen's 4K speed: the regression REPO-29 left open is not one

Outside units

• inference: the CPU reference's matvec across every core, bit-identical
• bench: --capture records one decode step into a Metal .gputrace; make capture
• metal-check: CAPTURE= records a micro-benchmark case into a .gputrace
• metal-check: capture names without spaces; the counter export's naming
• bench: saved prefixes, verify cost at depth, and the speed loop's A/B driver
• bench: saved-prefix I/O in 1 GiB pieces; a failed save keeps the run
• bench: --kernel-stats prints each Metal pipeline's limits
• inference: few-query verify attention through the split pass
• inference: register-fragment small-batch matmul tile
• inference: few-row generic matrices of a small batch on the multi-row matvec
• inference: the fragment tile for Q3_K, IQ3_S, Q4_0, PQ2_0, PTQ1_0
• inference: IQ4_NL small batches on...

nuclis v0.3.0

Models see images: Qwen3.8, Gemma 4, Muse Glimmer, and Bonsai 2 read them through their projectors on both the CPU and Metal, and the agent takes them by drop or /image. Gemma 4 E4B QAT joins, nuclis eval measures perplexity against the reference per family, and the agent's system prompt is rebuilt as sections measured on a playground task list. The terminal UI gets a repaint tick, banded diffs, and resize handling.

Models

• MODL-21 The vision contract, image input, and the Qwen3.8 projector on both executors
• MODL-22 Gemma 4 vision: the unified embedder (12B) and the SigLIP encoder (26B-A4B) on both executors; bidirectional image spans
• MODL-23 Muse Glimmer's windowed vision encoder on both executors; the image token cap
• MODL-24 Decode after an image: the Metal step's cache row and rotary position separated; the vision gate compares decode with one prefill
• MODL-25 Bonsai 2's projector through the Qwen3-VL adapter: Q8_0 and F16 weights
• MODL-26 Gemma 4's Metal verify rows carry the final soft-cap
• MODL-27 Gemma 4 E4B QAT: per-layer embeddings, shared KV, vision, draft head

Agent

• AGNT-11 A truncated tool call no longer bricks the session; reopened thought channels; copied bracket pieces (follow-up)
• AGNT-12 An empty tool result no longer aborts the turn (follow-up)
• AGNT-13 The system prompt as sections, measured: the playground task list, the guidelines that changed behaviour, the instructions file
• AGNT-14 Three measured fixes: the Qwen decoder keeps a value's trailing newline, read_file serves the first MiB, Enter steers a running turn
• AGNT-15 Images and text files in the chat: drop, /image, the chips, the projector turn, the detail row and preview, sessions
• AGNT-16 AGNT-14's follow-ups: the Muse value contract pinned and measured, steering that restarts a reasoning-only step, the guessed-path guideline measured and dropped
• AGNT-17 The playground fixes: blank output said, the paged-file rule, tool output in the transcript, a reasoning budget

Application

• APPS-14 Teacher-forced eval: perplexity against the reference's per-token run, the all-rows prefill, a gate per family
• APPS-16 Output budget: default 4096, cap 16384
• APPS-17 The catalogue at five entries; the registry in name order

Terminal

• TERM-10 Chat polish: the repaint tick, the frame, the operation rows, the banded diff, and the editor's shell and keys
• TERM-11 The region re-anchors on a resize from the terminal's cursor report; the preview fits above the region and is off under tmux
• TERM-12 A spinner row while the projector loads and images encode
• TERM-13 Exit keeps the transcript (in progress)

Repository

• REPO-12 The architecture guide follows the KV cache end to end; the inference guide rewritten as one narrative
• REPO-13 One specification: spec.md rewritten as a technical specification with the agent spec merged in
• REPO-14 The screenshot harness is the validation step for surface changes
• REPO-15 The CPU tier leaves the unit routine: it runs when a unit changes what the CPU reference computes, and before a release
• REPO-16 scripts/nuclis_mem_usage.py: a running process's memory split into GPU and CPU
• REPO-17 README rewritten, with a recorded agent session

Outside units

• repo: the screenshot harness runs the agent in a Ghostty window of its own (--direct)

All 62 commits

• 056122e chore(release): v0.3.0
• 84f1530 docs: README rewritten with a recorded agent session; close MODL-27 (REPO-17)
• bd96cbb feat(nuclis): the catalogue at five entries, the registry in name order (APPS-17)
• 4f2f977 feat(inference): Gemma 4 E4B QAT end to end, checks pending (MODL-27)
• d0ecb52 feat(agent): blank bash output said, the paged-file rule, Ctrl-O's output view, a reasoning budget at low (AGNT-17)
• a3253cb chore(scripts): nuclis_mem_usage.py splits a running process's memory into GPU and CPU (REPO-16)
• e9981d8 test(eval): a long-context perplexity tier; Gemma 4 12B passes at 4K, the 26B-A4B's -1.79 % is open (MODL-26 follow-up)
• d61685f perf(tokenizer): queued BPE merge for long pieces; Gemma lines no longer hit the work bound (...

nuclis v0.2.0

Speculative decoding arrives: batched verification with checkpoint and rewind of the recurrent state, Qwen3.8's draft head, Gemma 4's draft heads, and Muse Glimmer's DFlash drafter, each measured per family (on for Muse; off for Qwen and Gemma in this release). Three families join: Gemma 4 26B-A4B (mixture of experts), Bonsai 2 27B (ternary weights), and Muse Glimmer 30B. The agent learns Gemma's and Muse's native tool calls, survives truncated and empty tool results, and resumes sessions; the default context is 16K.

Models

• MODL-09 Gemma 4 26B-A4B: artifact pin, facts, adapter, CPU reference, Metal plan
• MODL-10 Gemma 4 26B-A4B: catalogue verdict, acceptance record, agent check
• MODL-11 Muse Glimmer 30B: artifact pin, facts, tokenizer, binding, CPU reference
• MODL-12 Muse Glimmer 30B: Metal plan
• MODL-13 Muse Glimmer 30B: profile (text, reasoning channel), catalogue, acceptance
• MODL-14 Catalogue entries ahead of their adapters; roadmap reordered around speculation
• MODL-15 Bonsai 2 27B accepted ahead of Muse: artifact pinned, facts read, three units planned
• MODL-16 Bonsai 2 27B: oracle, facts, ternary encodings, Hadamard transform, CPU reference
• MODL-17 Bonsai 2 27B: the Qwen plan on rotated weights, catalogue, acceptance
• MODL-18 Qwen3.8 draft head: the embedded prediction block on the CPU reference and the Metal plan
• MODL-19 Gemma 4 draft heads: the gemma4-assistant companion as a second GGUF, correct but a negative default at the plan's draft length
• MODL-20 Muse Glimmer DFlash drafter: the companion, the CPU reference and its trace, the Metal plan, a positive verdict

Engine

• ENGN-09 Primed sessions: prefill the prefix at startup, restore it on new
• ENGN-10 Muse Glimmer decode gap: the experiments accepted into the performance theme
• ENGN-11 Speculative state recovery: checkpoint, rewind, truncate, recover
• ENGN-12 Batched verification, speculative generation (greedy and sampled), the switch and the draft length, the benchmark record
• ENGN-13 Prompt commit at the plan's chunk and the batched drafter commit
• ENGN-14 Recovery without the whole-stack replay
• ENGN-15 Sampled acceptance on the GPU top-k readback
• ENGN-16 Draft proposal policy: the p_min early stop shipped, the adaptive length dropped
• ENGN-17 The speculative verdict: Qwen and Gemma off, Muse on; the full record and bench's true baseline
• ENGN-18 The speed loop: saved prefixes, verify cost at depth, interleaved A/B, and the decode-speed baseline (in progress)

Kernels

• KERN-09 Expert routing and gathered expert kernels (decode and prefill)
• KERN-10 Ternary matvec and matmul tiles, the Walsh-Hadamard kernel
• KERN-11 Small-chunk prefill matmul near the weight-bandwidth floor
• KERN-12 A multi-row matvec for 2–8 rows: the 2-row routing
• KERN-13 A GPU penalty kernel: the token history applied on the device before the top-k
• KERN-14 The wide 32×8 small-batch tile: measured, closed negative
• KERN-15 Split-K decode matvec: measured behind the single pass, closed negative
• KERN-16 Long-context prefill attention: register-level reuse measured 2–5 % at chunk sizes, closed negative
• KERN-18 Fused decode norms: −182…−192 dispatches per decode step shipped, the speed bars missed; closed below its target

Agent

• AGNT-08 Context budget: bounded results, in-turn elision, honest failure
• AGNT-09 Gemma 4 native tool calling
• AGNT-10 Muse Glimmer ATEM tool calling: rendering, decoding, fixtures
• AGNT-11 A truncated tool call no longer bricks the session; reopened thought channels; copied bracket pieces
• AGNT-12 An empty tool result no longer aborts the turn

Application

• **[APPS-06](https://github.com/tildaslashalef/nuclis...

nuclis v0.1.0

The first release: Qwen3.8-27B runs on Apple Silicon through an engine written in Zig with a Metal backend, from the GGUF bytes to the sampled token, every kernel checked against a pinned llama.cpp build. It passes the 32K-context acceptance run, with chunked prefill, tiled attention, chunkwise DeltaNet, and an F16 KV cache. Gemma 4 12B joins as a second family, models are pulled from the Hugging Face Hub with pinned digests, and nuclis agent puts a small coding agent (read, edit, and bash tools) on top in a terminal UI.

Models

• MODL-01 Qwen3.8 sampling profiles, min_p, penalties
• MODL-02 Model download through the huggingface package
• MODL-03 Models directory, catalogue, and registry
• MODL-04 Adapter registry
• MODL-05 Gemma 4 12B facts, binding, CPU reference, tokenizer
• MODL-06 Gemma 4 12B Metal plan
• MODL-07 Gemma 4 12B profile, catalogue, acceptance, new-model guide
• MODL-08 Q4_0 path and the QAT catalogue entry

Engine

• ENGN-01 Library engine API: the engine seam moved into inference
• ENGN-02 Chunked prefill with the batched matmul
• ENGN-03 Causal tiled attention for prefill
• ENGN-04 Chunkwise DeltaNet (WY form)
• ENGN-05 Matmul tile ceiling: specialized half-operand 64×64 tiles
• ENGN-06 Session snapshot and restore
• ENGN-07 32K acceptance run and benchmark record
• ENGN-08 Long-context prefill attention

Kernels

• KERN-01 Specialized matvec for Q4_K, Q5_K, Q6_K, IQ4_XS
• KERN-02 Per-kernel GPU profile, Observer check/layer split
• KERN-03 Specialized matvec for Q3_K, IQ3_S (IQ4_NL deferred)
• KERN-04 Merged projections via a segment table
• KERN-05 Per-block cost of the specialized kernels
• KERN-06 GPU partial top-k for sampled decoding
• KERN-07 F16 KV cache as a session layout option
• KERN-08 Flash-decoding attention

Agent

• AGNT-01 Completion events and the tool seam
• AGNT-02 The agent loop: state machine, provisional parser, read tools
• AGNT-03 Read tools and bash
• AGNT-04 Mutations and diffs
• AGNT-05 Qwen tool rendering and pinned fixtures
• AGNT-06 Tool-call decoding and the profile-driven loop
• AGNT-07 Polish, resume, and the agent plan closed

Application

• APPS-01 --think reasoning effort
• APPS-02 nuclis chat playground
• APPS-03 Engine configuration ~/.nuclis/nuclis.json
• APPS-04 Styled command output
• APPS-05 config init registers the catalogue; chat → agent

Terminal

• TERM-01 Agent terminal surface
• TERM-02 Grapheme-correct width, wrapping, and cursor motion
• TERM-03 OSC 8 hyperlinks in markdown
• TERM-04 Focus tracking and turn-complete notifications

Repository

• REPO-01 Repository restructure: standalone src/ + inference/
• REPO-02 Reference oracle promoted to committed tests/fixtures/
• REPO-03 Version wiring: nuclis --version from the manifest
• REPO-04 Comment and identifier hygiene

Outside units

• import nuclis v0.1

All 2 commits

• 1a31568 chore(release): v0.1.0
• cf76bc0 feat: import nuclis v0.1

---

The attached build

macOS on Apple Silicon, release mode, with the Metal backend. It is not
signed or notarized — there is no Apple Developer ID for this project — so
macOS quarantines it on download. After unpacking:

shasum -a 256 -c SHA256SUMS               # optional, and worth it
gh attestation verify nuclis-*-aarch64-macos.tar.gz -R tildaslashalef/nuclis  # built here
tar -xzf nuclis-*-aarch64-macos.tar.gz
xattr -d com.apple.quarantine nuclis-*/nuclis
./nuclis-*/nucl...