Skip to content

v0.3.0 — Greedy verification and Qwen3.5 state recovery

Choose a tag to compare

@AkiYangDev AkiYangDev released this 24 Oct 12:58

这一版讲清候选验证之后的状态处理:哪些输出可以提交,哪些 KV 还能读,Qwen3.5 的卷积窗口和递推矩阵怎样恢复。

三个 CPU 入口按顺序运行,无需下载权重:

python -m minideepinfer.spec_demo
python -m minideepinfer.spec_demo --hybrid
python -m minideepinfer.spec_demo --hybrid --draft lookup

先用受控候选观察拒绝与 bonus,再检查混合状态的快照和重算,最后让 lookup 只根据已提交历史提出候选。独立候选模式在生成完成后才运行 greedy 对照。

Ascend 910C 上的真实 Qwen3.5-2B FP32 输出和冷缓存状态检查均通过。lookup 短例子接受 0/5,target 工作量由 10 forward / 14 输入 token 增至 12 / 21;没有计时或提速结论。这个结果和快照、查找、重算成本一起保留。

189 项测试在 Transformers 5.12.1 下全部通过;4.46.3 为177通过、12跳过;基础环境为70通过、119模型检查跳过。三个 CPU 模式均通过安装后的仓库外检查。本次发布整理不改执行源码,复用已有实机报告,没有重新加载模型。

安装方法、预期输出和六份报告的固定版本见完整说明。附件包含 wheel、源码包和 SHA256SUMS;尚未发布到 PyPI。

English

v0.3.0 follows a candidate through greedy verification, output commit, KV rollback, and Qwen3.5 state recovery. The three CPU commands above need PyTorch but no checkpoint. The lookup draft reads committed history only; its independent greedy baseline runs afterwards.

Preserved real Qwen3.5-2B FP32 checks on Ascend 910C match greedy output and cold-cache state. Lookup accepted none of five proposals and increased target work from 10 forwards / 14 input tokens to 12 / 21. There is no timing or speedup claim. Snapshot, lookup, replay and reference costs remain separate. This is a synchronous teaching path; it does not implement learned drafts, MTP/DSpark, probability-preserving sampling, SWA Bounded Replay or cross-round speculation.

All 189 tests pass with Transformers 5.12.1, and all three installed CPU modes run outside the checkout. Release preparation reuses preserved hardware evidence without changing execution source or reloading weights. See the release walkthrough for setup, scope and evidence. Assets include a wheel, source distribution and SHA256SUMS; this release is not on PyPI.