Home

GenerelSchwerz llama.cpp fork

Join the OptLlama Discord

This fork adds memory controls, CUDA MoE acceleration, speculative decoding work, and reproducible benchmark evidence to llama.cpp.

Start here

Goal	Guide
Choose a branch	Branch table
Build the CUDA MoE cache	moe-cache
Configure every MoE flag	MoE flags and setup
Run on native Windows	Windows WDDM setup
Build with Docker Compose	Docker Compose for large MoE models
Choose a VRAM target	4, 8, 12, and 16 GB guides
Reproduce this fork's performance	Notable runs
Compare stock and fork builds	Benchmark comparison
Find one fork feature	Feature index

Important

CUDA REQUIRED FOR MOE CACHING: Use an NVIDIA GPU and build with -DGGML_CUDA=ON. Other branches and ordinary llama.cpp placement can use their supported non-CUDA backends.

Important

WINDOWS USERS: Read Windows WDDM setup before enabling the MoE cache. Use matched fork DLLs and set a host-pinning budget that Windows actually grants.

Choose a branch

Branch	Choose it for	Main additions	Docs
llama/main	Conservative upstream tracking	Pinned host KV, partial target/draft KV residency, phase-aware workspace, bounded MTP state	Overview - flags
llama/dev	Newer memory work	Everything above, plus compact causal masks and live-context workspace	Overview - flags
beellama/main	BeeLlama plus fork memory controls	BeeLlama features, fork KV/workspace controls, MTP validation	Overview - flags
moe-cache	CUDA MoE development	GPU expert cache, grouped decode, drafting, MTP, overlap, prefetch, partial host pinning	Overview - flags

ggml-org/llama.cpp
    |
    +-- llama/main
            |
            +-- llama/dev
                    |
                    +-- moe-cache

BeeLlama
    |
    +-- beellama/main

Do not mix commits, state files, executables, or shared libraries between the BeeLlama and llama.cpp lineages.

Tune safely

1. Run the exact binary's --help after each update.
2. Start with --load-mode none when the cold expert source fits system RAM.
3. Use bounded host pinning when the platform cannot pin the complete source.
4. Budget model weights, KV, workspace, recurrent state, draft models, and expert-cache pools separately.
5. Confirm coherent output before measuring speed.
6. Record prompt depth, generated tokens, parallelism, prefill, decode, loaded VRAM, and peak VRAM.

The upstream llama.cpp documentation remains the authority for standard llama.cpp options. This wiki covers fork additions and measured configurations.

Feature index - Hardware guides - Evidence - Discord