-
Notifications
You must be signed in to change notification settings - Fork 17
Draft Context Namespace
LLAMA_CONTEXT_TYPE_DRAFT distinguishes a separately loaded speculative draft context from the main target context and the integrated LLAMA_CONTEXT_TYPE_MTP context. The namespace prevents cache plans, graph authority, and execution statistics from being reused under the wrong owner.
Common speculative setup assigns the draft type automatically when it constructs a separate model-backed drafter. Existing target and MTP defaults are unchanged. Applications normally do not need to set the type themselves.
The distinction becomes important once backend resources survive more than one graph call. A target graph and draft graph can have similar tensor shapes but different model ownership, cache slot pools, and lifecycle. Treating them as interchangeable could point a captured graph or cached plan at the wrong weights.
There is no CLI flag. This is a public/runtime integration helper for the grouped MoE branches.
| Branch | Context type | Automatic assignment |
|---|---|---|
moe-cache |
include/llama.h |
common/speculative.cpp |
The helper was introduced with the generic grouped architecture reviewed in PR 56.
GenerelSchwerz llama.cpp
- Home
- Discord community
- Contributors
- Complete feature index
- Hardware setup guides
- Owner-verified evidence
- Notable runs
- Benchmark comparison showcase
- BeeLlama Main
- Llama Main
- Llama Dev
- MoE Cache
Feature groups
- Memory placement and workspace
- Validation and diagnostics
- CUDA MoE cache and helpers
- Grouped MoE drafting
Source branches