Repository navigation
v0.4.0 — Follow draft proposals through verification and recovery
v0.4 follows a draft proposal through verification, correction, cache recovery and the next round. The draft and target each keep their own cache, using the existing synchronous greedy verification path.
With compatible CPU PyTorch, run the installed package without checkpoints:
python -m minideepinfer.model_draft_demo --cpu-example rejection --trace-rounds --width 3 --output outputs/draft-rejection.jsonThe untrained random models produce rejected candidates without answer lookup or candidate injection. Watch the draft move from six computed inputs back to four valid ones, then consume the correction to reach five. Omit --cpu-example rejection for an all-accepted comparison.
- Round observations show actual candidates, model inputs, commits, logical cache lengths and cleanup. They add no model calls and run separately from timing.
- Tokenizer compatibility is checked before real weights load. Width-1 draft snapshots are skipped when no tentative candidate is fed back.
- Failed drafts cannot be reused. Interruptions and cleanup errors preserve failure reports, including unknown target cleanup as
NOT_OBSERVED. - All 214 tests pass with Transformers 5.12.1; 202 pass and twelve skip with 4.46.3; the base environment passes 75 and skips 139. Both CPU walkthroughs match their preserved reports when installed outside the checkout.
- The Ascend 910C FP32 release check uses Qwen3.5-0.8B → 2B: widths 1/3 accept 4/5 and 6/7, match independent greedy output, pass sixteen cold-cache comparisons and return all fourteen slots per model. Each existing checkpoint loads once; the full program makes fifty forward calls. No new performance timing or speedup claim.
This is a teaching implementation, separate from cross-round generation. It does not implement MTP/DSpark, probability-preserving sampling or SWA Bounded Replay. Earlier slower and small-effect timing results retain their original source revisions. Old releases and raw reports are preserved.
Assets: wheel, source distribution and SHA256SUMS. This release is not uploaded to PyPI.
中文
这一版把“两模型猜错后怎样继续”串成可运行的逐轮演示:候选来自小模型实际预测,打印 target 的验证与提交,以及下一轮 draft 恢复前缀的过程。CPU 不需要权重;真实 0.8B/2B 已完成 Ascend 910C 正常生成与状态验收。异常后禁止复用半份缓存,失败报告保留中断、清理错误和未知状态。