Llama Dev

llama/dev

llama/dev adds newer attention-memory work to llama/main. Documented checkpoint: 4d60222d6.

Note

BACKEND SUPPORT VARIES: Live-context sizing uses backend-generic contracts. Compact masks currently have CPU and CUDA consumers; other backends keep dense masks.

Added features

• Compact causal masks, selected automatically for eligible Flash Attention graphs
• Live-context workspace, enabled with --live-context-workspace

The branch inherits every llama/main control.

How they behave

Compact masks replace a dense causal-prefix mask with one boundary index per query row. Multiple streams, sliding-window attention, ALiBi, reordered KV cells, gaps, model-specific edits, or unsupported backends keep the dense mask.

Live-context sizing reserves supported attention workspace from the padded live KV extent and grows it as context grows. It does not shrink persistent KV or model weights. Combine it with --phase-aware-workspace when both live context and prompt/decode geometry matter.

Measure startup, prompt peak, deep-context decode, and later-turn regrowth. Smaller startup allocation does not guarantee a smaller final peak or faster decode.

Build

git clone --branch llama/dev https://github.com/GenerelSchwerz/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j 20
./build/bin/llama-server --help

Flags and setup - llama/main - Feature index - Home