Sparse Recurrent Snapshots

Sparse recurrent snapshots

Sparse recurrent snapshots let an MTP verification graph write the recurrent state for one selected token boundary instead of materializing every rollback boundary. This is the backend/model contract that makes capped MTP recurrent planes correct.

Contract

Support is proven at three levels: the model graph declares that its recurrent topology is compatible, the memory object exposes selected snapshot storage, and every backend executing the relevant gated-delta-net operations accepts the sparse snapshot parameters. The server enables sparse mode only after all three agree.

The current model declaration is deliberately narrow. On the llama/main family, the shipped declaration covers Qwen 3.5 and Qwen 3.5 MoE GDN graphs. This is not a general promise for every recurrent architecture. Unsupported models or mixed backend graphs fail closed when capped replay is requested.

Note

Backend scope: At the audited published tips, beellama/main exposes the required sparse operation on NVIDIA CUDA. Current llama/main and llama/dev support CPU, CUDA, and Vulkan. The audited MoE tips predate the Vulkan port and support CPU/CUDA. Metal, SYCL, and other backends are not claimed.

Branch implementations

Branch	Model/memory contract	Backend implementation
beellama/main	src/llama-memory-recurrent.cpp, src/models/delta-net-base.cpp	CUDA
llama/main	src/llama-memory-recurrent.cpp, src/models/models.h	CPU, CUDA, Vulkan
llama/dev	src/llama-memory-recurrent.cpp, src/models/models.h	CPU, CUDA, Vulkan
moe-cache	Inherited pre-Vulkan contract at its audited tip: src/llama-memory-recurrent.cpp	CPU and CUDA

Design history: PR 37. Vulkan support was added by commit 5903044bc.

Feature index