Graph Execution Certificates

Graph execution certificates

Graph execution certificates attach short-lived semantic information to one scheduler compute call. They tell an optimizing backend whether a graph belongs to the main target, a separate draft context, or integrated MTP, and whether its rows are independent, sequential, or speculative.

Contract

Callers use ggml_backend_sched_graph_compute_ext() or its asynchronous counterpart with a ggml_graph_execution_certificate. Existing scheduler APIs remain available and produce ordinary uncertified execution.

The scheduler stamps source and split graph identities only for the duration of each backend callback. A backend must validate and copy the fields it needs synchronously and must never retain the certificate address. Missing, invalid, or contradictory certificates do not grant an optimized path.

The llama runtime automatically certifies eligible independent decode batches. The CUDA MoE planner combines this semantic proof with a complete candidate manifest and backend capability checks before grouped execution. This avoids guessing row meaning from tensor shapes alone.

Note

The contract is backend-generic, but its metadata ABI is private to the fork implementation. The shipped feature consumer is currently the CUDA grouped MoE path.

Branch implementations

Branch	API and scheduler	Runtime producer
moe-cache	ggml/src/ggml-backend-moe.h, ggml/src/ggml-backend.cpp	src/llama-context.cpp

Design/review history: PR 56.

Feature index - Draft context namespace