Skip to content

Draft Context Namespace

Rocco A edited this page Sep 14, 2026 · 2 revisions

Draft context namespace

LLAMA_CONTEXT_TYPE_DRAFT distinguishes a separately loaded speculative draft context from the main target context and the integrated LLAMA_CONTEXT_TYPE_MTP context. The namespace prevents cache plans, graph authority, and execution statistics from being reused under the wrong owner.

Behavior

Common speculative setup assigns the draft type automatically when it constructs a separate model-backed drafter. Existing target and MTP defaults are unchanged. Applications normally do not need to set the type themselves.

The distinction becomes important once backend resources survive more than one graph call. A target graph and draft graph can have similar tensor shapes but different model ownership, cache slot pools, and lifecycle. Treating them as interchangeable could point a captured graph or cached plan at the wrong weights.

There is no CLI flag. This is a public/runtime integration helper for the grouped MoE branches.

Branch implementations

Branch Context type Automatic assignment
moe-cache include/llama.h common/speculative.cpp

The helper was introduced with the generic grouped architecture reviewed in PR 56.

Feature index - Graph execution certificates

Clone this wiki locally