BeeLlama v0.4.0 + Q2_0 ternary CUDA (Blackwell / sm_120 build)

BeeLlama v0.4.0 + Q2_0 (ternary, group-64) CUDA — Blackwell build

This is an unofficial build of Anbeeld/beellama.cpp v0.4.0 with the CUDA weight
path for GGML_TYPE_Q2_0 (group-64 ternary, id 42), so ternary GGUFs such as
PrismML's Ternary-Bonsai-27B group-64 run on the GPU. Kernels are ported from
upstream ggml-org#25707 (@khosravipasha) — see PR Anbeeld#96 for provenance.

What's in the zip

Windows x64 runtime: all .dll plus the llama-*.exe tools (server, cli,
perplexity, bench, quantize, …). Test executables are omitted.

Requirements

• NVIDIA Blackwell GPU, compute capability 12.0 (RTX 50-series). The DLL is
  built only for sm_120; it will not run on older architectures.
• CUDA 12.8 runtime on PATH (cudart64_12.dll, cublas64_12.dll,
  cublasLt64_12.dll). Install the CUDA 12.8 toolkit/runtime, or place those
  DLLs next to the executables. They are not bundled here.
• Windows 10/11 x64.

Verification

GPU perplexity matches the CPU reference within FP tolerance on a group-64 GGUF:
6.7145 (GPU, -ngl 99) vs 6.7143 (CPU, -ngl 0) over 6x512-token chunks — both
decode (MMVQ) and prefill (MMQ) paths are correct.

Quick start (Ternary-Bonsai-27B group-64)

llama-server.exe -m Ternary-Bonsai-27B-Q2_g64.gguf ^
  -ngl 99 -fa on -c 65536 -ctk f16 -ctv f16 --port 8109

KV cache: use f16 up to ~128K (measured)

On Blackwell, dequant-free f16 KV is both fastest and highest quality until it
no longer fits. Decode t/s at 32K depth (llama-bench, Ternary-Bonsai-27B):

f16      39.1     <- fastest
q8_0     35.3
kvarn3   34.3
kvarn4   33.4

Bonsai f16 KV is ~64 KiB/token: 4 GiB @ 64K, 8 GiB @ 128K, 16 GiB @ 262K.
Use KVarN only near the ceiling (192-262K) where f16 no longer fits, and prefer
symmetric kvarn3/kvarn3 there. Within KVarN a higher number is not faster —
all widths pay the same normalization cost; more bits just means more bytes.

DFlash on v0.4.0

v0.4.0 uses upstream DFlash. It is a real speedup on predictable content
(code, structured text, tool calls) and roughly neutral on free prose:

Ternary-Bonsai-27B, greedy, --spec-draft-n-max 3, f16 KV:
  code   0.69 acceptance   81 t/s   (~1.85x over 44 t/s base)
  prose  0.21 acceptance   44 t/s   (= base)

--spec-draft-n-max 3 is the sweet spot. Higher depth or the adaptive
controller (uncapped n_max=15) draft too eagerly at low acceptance and end up
slower.

llama-server.exe -m Ternary-Bonsai-27B-Q2_g64.gguf ^
  -ngl 99 -fa on -c 8192 -ctk f16 -ctv f16 ^
  --spec-type draft-dflash -md DRAFTER.gguf -ngld 99 --spec-draft-n-max 3

Adapting an old DFlash drafter to v0.4.0

v0.4.0 only accepts upstream-schema drafters (general.architecture = dflash).
Older fork-era heads declare dflash-draft and will fail to load with
unknown model architecture: 'dflash-draft'. remap_dflash.py (attached)
rewrites the metadata and tensor names in place, no re-training or
re-quantization — tensor data is copied byte-for-byte:

python remap_dflash.py OLD-DFlash.gguf NEW-dflash-upstream.gguf

It maps: arch dflash-draft -> dflash; key prefixes dflash-draft. ->
dflash.; dflash.block_size, tokenizer.ggml.mask_token_id,
dflash.target_layers; tensors dflash_fc -> fc, dflash_hidden_norm ->
enc.output_norm, post_attention_norm -> ffn_norm. Requires the gguf
Python package. Validate by draft acceptance, not just that it loads.

License

MIT, same as upstream. This build adds nothing to the license terms; see the
LICENSE in the repository.