Feature Index

Fork feature index

Find every shipped fork feature here. Use the normal llama.cpp documentation or BeeLlama documentation for upstream behavior.

Published tips audited for this index:

Branch	Audited tip
beellama/main	4b86269fdf
llama/main	5903044bc
llama/dev	4d60222d6
moe-cache	925933801

Memory placement and workspace

Feature	Branches	Backend scope
Pinned host KV	All four	Capability-dependent; CUDA is the best-qualified path
Independent recurrent-state placement	All four	Capability-dependent
Partial target KV GPU residency	All four	Capability-dependent
Independent draft KV GPU residency	All four	Capability-dependent
Canonical host-KV quantization	All four	Requires a supported accelerator store route
Independent draft ubatch	All four	Backend-generic; MTP deliberately restricts it
Phase-aware workspace	All four	Backend-generic allocator contract
Live-context workspace	beellama/main, llama/dev, and descendants of llama/dev	Backend-generic sizing; physical release depends on backend allocation behavior
Compact causal masks	llama/dev, moe-cache	CPU and CUDA at the audited tips
Capped MTP recurrent planes	All four	Requires the sparse snapshot contract below
Sparse recurrent snapshots	All four	Model- and backend-gated; CUDA on the audited BeeLlama tip, CPU/CUDA/Vulkan on current llama/main and llama/dev, and CPU/CUDA on the audited MoE tip

Experimental row prefetch and server overlap

All three controls now ship on moe-cache at 925933801.

Feature	Branches	Scope
Lazy PLE row prefetch: --ple-prefetch	moe-cache	Generic CPU gather advice; Linux measured, Windows experimental/performance-unvalidated; no CUDA requirement
Decode overlap: --decode-overlap	moe-cache	CUDA backend-sampled decode, first MTP draft-step overlap, and eligible Flash Next staged host inputs
Decode boundary overlap: --decode-boundary-overlap	moe-cache	Opt-in CUDA graph updates and eligible boundary preparation overlap; disabled by default

Validation and diagnostics added to BeeLlama

Feature	Branches	Scope
Feature/performance validation toolkit	beellama/main	Developer tool, no runtime dependency
CUDA VMM allocation telemetry	beellama/main	CUDA required
Perplexity full-batch output capacity	beellama/main	Tool correctness fix

CUDA MoE cache and generic helpers

Important

CUDA REQUIRED: MoE cache execution requires an NVIDIA CUDA build. Certificate, namespace, and sampler helpers contain no CUDA kernels but support this path.

Feature	Branches
CUDA MoE expert cache	moe-cache
MoE host sources and partial pinning	moe-cache
MoE prefetch and overflow staging	moe-cache
Cached MoE prefill	moe-cache
Grouped MoE decode	moe-cache
MoE candidate manifests	moe-cache
Graph execution certificates	moe-cache
Draft context namespace	moe-cache
Dense sampler penalty counts	moe-cache
MoE CUDA graph reuse	moe-cache
MoE cache ownership and parallelism	moe-cache
MoE cache diagnostics	moe-cache
Parallel mapped-MMQ compilation	moe-cache
Resident MoE auxiliary banks	moe-cache

Grouped MoE drafting

Important

CUDA REQUIRED: The fork adapts upstream speculative methods to the CUDA MoE cache. Upstream still defines the drafting algorithms.

Feature	Branches
Grouped MoE drafting	moe-cache
Independent draft MoE cache: --spec-draft-moe-expert-cache-size	moe-cache
Atomic speculative verification	moe-cache
Per-sequence draft limits	moe-cache
Retained MTP state	moe-cache
Speculative MoE graph variants	moe-cache

Reading the source links

Use moe-cache at 925933801 as the documented behavior. Immutable older commits only preserve design or benchmark history.

Home