v0.2.0 — a second model family

Godwit runs a second model family. GPT-OSS-120B, GPT-OSS-20B and Qwen3-30B-A3B, from one binary, switchable at runtime from the dashboard.

That was the claim v0.1.0 could not support. The 20B ran first time with no change beyond declaring its spec, but it shares everything with the 120B — same tensor names, same RoPE, same activation, same tokeniser. Qwen3 shares almost nothing, and it broke seven things rather than the five predicted.

Models

	GPT-OSS-120B	GPT-OSS-20B	Qwen3-30B-A3B
Layers × experts	36 × 128	24 × 32	48 × 128
Active per token	4	4	8
Install	59.2 GiB	11.2 GiB	16.1 GiB
Resident	5.7 GiB	4.1 GiB	2.5 GiB
Decode	1.4 tok/s	2.8–3.2 tok/s	2.3 tok/s

Measured on a 13" M4 MacBook Air, 16 GB, base 256 GB SSD.

What Qwen3 needed

Per-expert tensors instead of stacked ones, QK-norm, plain SwiGLU, plain RoPE, ChatML, no attention sinks, no biases — and MXFP4 written rather than copied, because Qwen3 ships BF16 where GPT-OSS ships already-quantised weights.

Two of those would have run without complaining:

• A zero attention sink is not the identity. The sink enters the softmax denominator as exp(sink - max), so a sink-free model given a zero sink gains a whole unit of mass on every row. It has to be compiled out.
• MXFP4 encoding has to be checked against something. godwit check-mxfp4 decodes OpenAI's shipped weights and re-encodes them, demanding byte equality across 2,073,600 blocks. It caught a floored exponent and a dropped sign bit, either of which would have written 15 GB of quietly wrong weights.

One bug worth reading about

Qwen3 first produced fluent English with correct facts and then could not stop — it would say Paris, doubt itself, and re-derive the answer until the token budget ran out. Every piece of evidence pointed at quantisation: MXFP4 costs Qwen3 18.9 dB where it costs GPT-OSS nothing, and the install verified at 18.5 dB against the checkpoint, so the weights on disk were provably right.

The fix looked like int8 experts: a 30 GB install, another two hours, and deleting the 120B to make room. All three would have been spent on a wrong diagnosis.

A NumPy reference for one attention block settled it in an hour. Attention was off by 4.9 against a bound of 0.013 — not a quantisation signature at all. The QK-norm weight was declared half in the Metal kernel where every other small trunk tensor is bfloat.

Dashboard

• Model picker in the header — switch between installs, or download one that is missing
• Reasoning streams as it arrives, rather than appearing at the end; a turn that spends its budget thinking says so instead of showing an empty reply
• Range maps build from a button, with per-topic progress
• Range map counts now require 24 activations before crediting a topic label. On the 120B that excludes 40% of plotted experts and takes the python count from 516 to 248 — an earlier README reported the larger number
• New palette: cyan on slate

Also

• serve accepts the sampling flags the README always claimed it did
• The installer refuses an unrecognised repository instead of silently falling back to GPT-OSS-120B
• 82 tests, up from 54

Full detail in the README and docs/DESIGN.md.

v0.1.0 — first working release

Run GPT-OSS-120B on a 16 GB Mac. First working release: it installs, it
generates, and everything it claims has been measured.

What works

• Install — streams the 59 GiB checkpoint from Hugging Face and repacks it
  into a .gwt directory without ever materialising a shard. Verified
  byte-exact against the source.
• Chat — terminal or one-shot, with temperature, top-k, top-p, repetition
  penalty and seeding. No Python; the tokeniser is Swift and matches tiktoken
  exactly on 14 cases including emoji with ZWJ sequences, CJK and Cyrillic.
• Dashboard — godwit serve gives chat, a live grid of all 4,608 experts
  lighting up as the router picks them, and a range map of what each expert
  specialises in. No dependencies; the binary serves the whole interface.
• Measurement tools — profiling, kernel A/B testing that survives thermal
  drift, routing traces, and NumPy reference checks at every numerical stage.

Performance

Measured on a 13″ M4 MacBook Air, 16 GB, base 256 GB SSD:

Decode	1.4–1.5 tok/s, flat across the generation
Prefill	~2.2 tok/s
Time to first token	10–13 s
Resident	2.12 GiB trunk + 3.56 GiB expert cache
GPU busy	17.6% of wall time

This is device speed, not an inefficiency. Expert reads are 71.6% of decode
and the read path runs at what this SSD delivers. Small Apple SSDs use fewer
NAND dies and are genuinely slower — this one reads ~2 GiB/s where 512 GB–2 TB
modules reach 3–6. The same code should reach roughly 3–5 tok/s on a Mac with a
larger SSD. That has not been verified, because no such machine was
available.

Known limits

• One model family. About 78% of the Swift is model-agnostic, but tensor
  naming, YaRN-only RoPE and the MXFP4 assumption are not.
• Slower than reading speed. GPT-OSS-20B fits in the same machine at 30–50
  tok/s under MLX; if that is good enough for your task, use it instead.
• The range map's per-topic counts overstate specialisation. 73% of the experts
  labelled python fired fewer than twenty times across the probes. Roughly 150
  of the 545 are defensible. More probe samples would fix it.
• Conversation history is re-encoded each turn, so the KV cache is rebuilt
  rather than reused across turns.

Correctness

Every numerical stage is checked against a NumPy reference built from the same
installed bytes: MXFP4 decode byte-exact against OpenAI's tensors, expert
feed-forward to 3e-06, attention to 3e-04 RMS, and a full layer with expert
selection matching exactly. 54 tests, green on CI.

Requirements

Apple Silicon Mac, macOS 26+, Swift 6.2+, 16 GB memory, ~60 GB free on fast
internal storage. Xcode is not required.

---

Full notes: design ·
results ·
site