Repository navigation
Bonsai 2
- Add Bonsai 2 2B, 4B, 9B and 27B to Get models, with matching vision projectors. All four sizes support text and single-image input on supported Windows CPU/CUDA routes. (#814)
- The default CUDA projection path now quantizes rotated activations to Q8 and uses DP4A integer dot products. On an RTX 3060 Laptop GPU this gives 2.07–2.47x higher ordinary throughput for 2B/4B/9B and 1.28–1.66x for 27B, compared with the previous float path. The float path stays available through
CAMELID_BONSAI2_CUDA_PROJECTION=tiled. - 27B streams packed weights from pinned host memory when VRAM is short. An independent 2B drafter is available as an opt-in; it helps 27B on counting and code prompts and slows smaller sizes, so drafting stays off by default.
Faster Bonsai decoding on Apple Silicon
- Q1_0 matrix multiplies now launch the right number of GPU threadgroups instead of eight times too many. (#817)
- The Q1_0, Q2_0 and PQ2_0 Bonsai models now use the GPU sampling fast lane, so no CPU work sits between tokens for greedy and seeded temperature sampling.
- Measured on Bonsai-4B-Q1_0 on an Apple M3, 200-token replies: greedy 38.5 → 47.4 tok/s, seeded temperature 35.8 → 47.3 tok/s,
top_p+top_k41.0 → 44.5 tok/s,top_palone 35.3 → 38.6 tok/s. Greedy andtop_p/top_koutput is byte-identical to before. Seeded temperature-only sampling now draws from the GPU sampler, so the text for a given seed differs from 0.8.0.
Observatory and UI
- Add Star Nursery, a new default Observatory renderer in which every decoded token ignites a star, so a finished reply becomes a constellation. Flow Bench and Neural Field remain available. (#815)
- The Observatory now runs edge to edge, with the instrument rail floating over the canvas on wide screens. (#816)
- Fix the System and API tiles reporting "Online" while the engine is offline, fix in-app hash links and the Back button, and fix composer buttons overlapping at phone width.
- While the engine-offline banner is showing, hide the duplicate connection errors that restated it.
Validation
Cross-platform CI passed on each merged pull request. The release workflow built every package; the macOS jobs moved to the macos-latest runner because GitHub retired the macos-14 image (#819). All five package digests match their published checksums. Bonsai 2 measurements and evidence are in docs/BONSAI2.md and qa/bonsai2/cuda-q8. The Apple Silicon Bonsai changes were measured on 4B Q1_0 only; the 8B and 27B rows and the Q2_0/PQ2_0 rows were not run locally.
Full Changelog: v0.8.0...v0.8.1