Skip to content

speculative-simple : add checkpoint support#22227

Merged
ggerganov merged 2 commits intomasterfrom
gg/spec-simple-add-ckpt-support
Apr 22, 2026
Merged

speculative-simple : add checkpoint support#22227
ggerganov merged 2 commits intomasterfrom
gg/spec-simple-add-ckpt-support

Conversation

@ggerganov
Copy link
Copy Markdown
Member

Overview

cont #19493

  • Add speculative checkpoints to speculative-simple
  • Avoid cloning the sampler in llama-server when not necessary

Additional information

Sample command:

make -j && ./bin/llama-speculative-simple -hf ggml-org/Qwen3.5-35B-A3B-GGUF:Q8_0 -md ~/models/qwen3.5-0.8b-base/ggml-model-q8_0.gguf -p "Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" -n 256 --draft 32 --temp 0 --top-k 1 --seed 42 -ngl 99 -ngld 99

Requirements

@ggerganov ggerganov requested a review from a team as a code owner April 21, 2026 18:25
@ggerganov ggerganov merged commit bcb5eeb into master Apr 22, 2026
48 of 49 checks passed
@ggerganov ggerganov deleted the gg/spec-simple-add-ckpt-support branch April 22, 2026 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants