v0.3.2-ternary-bonsai.2 — native Q2_0 restored, DFlash works with Bonsai

Makes the fork a drop-in replacement for upstream BeeLlama again, and documents that speculative decoding does work with Ternary Bonsai.

Based on BeeLlama preview-v0.3.2 (commit a620cbd). The version here names that base; the .2 suffix counts revisions of this fork. Upstream version numbers belong to Anbeeld.

The two 2-bit types now coexist

The first release repurposed Q2_0 for PrismML's ternary format, which silently broke BeeLlama's own 2-bit type — same name, incompatible layout. They are now split by on-disk id:

type	id	name shown	weights	KV cache
BeeLlama native	53	q2_0	yes	yes
PrismML ternary (Bonsai)	42	q2_0_t	yes	no

The native type keeps the name q2_0, so existing models and existing command lines (-ctk q2_0, -ctk kvarn2, …) behave exactly as upstream.

The ternary format has no runtime quantizer — PrismML encodes offline — so it cannot back a KV cache. Requesting a 2-bit KV cache gives you the native planar type, as before.

Fixes

Four independent breakages in the previous release, all on the KV path:

• the planar 2-bit encoder wrote into 128-element ternary blocks
• the flash-attention K/V helpers read the wrong block geometry
• -ctk kvarn2 fell back to the ternary type on SWA layers
• case GGML_TYPE_Q2_0_B labels were missing in convert.cu / getrows.cu, leaving unreachable code after a return

If you are on the previous release, upgrade — with any 2-bit KV option it produced garbage without an error.

Speculative decoding: use the stock Qwen3.6-27B DFlash head

A DFlash drafter trained for plain Qwen3.6-27B accelerates Ternary Bonsai too. The head is tied to the architecture, not to the target's quantization — it ships no token_embd/output tensors (shared from the target at runtime) and cross-attends to the target's hidden states.

Pass --spec-draft-n-max 3 explicitly. DFlash's adaptive controller settles on 15, which suits ordinary quants accepting ~30% of drafted tokens. Bonsai accepts far less, so long draft blocks are mostly wasted:

--spec-draft-n-max	decode	acceptance
2	62.5 t/s	69.4%
3	64.4 t/s	63.1%
4	60.3 t/s	52.6%
8	55.7 t/s	34.4%
15 (adaptive default)	51.3 t/s	15.1%

Baseline without a drafter: 47.4 t/s. So n_max=3 is +36%, versus +8% if you let the controller decide. 3 happens to be llama.cpp's own default for the flag.

Measured at 32K context, short prompt, greedy, one card, one drafter. Acceptance normally falls as context grows, which would push the optimum shorter still — not measured.

Verification

The ternary path is unchanged, and this is checked rather than assumed. Perplexity on the same corpus and settings (-c 4096 -fa on --chunks 12):

PrismML reference:  3.1125 +/- 0.04339
this build:         3.1125 +/- 0.04339

A wrong bit layout fails silently, so a digit-for-digit match is the check that actually rules it out.

Type 53 is restored from the pre-port sources rather than tested — no model in that format was available here. Its code is the upstream implementation retargeted to the split type; reports welcome.

Binary

Windows x64, CUDA 12.8, built natively for sm_120 (Blackwell). Other architectures: build from source.

Known cosmetic issue inherited from upstream: the loader prints unknown type q2_0_t. BeeLlama never wired its own 2-bit types into the ftype table either; behaviour is unaffected.

MIT, same as upstream. Built on Anbeeld/beellama.cpp and PrismML-Eng/llama.cpp.