v0.5.0 — Stream and sample with two models

v0.5.0 brings custom text, streamed output and probability sampling to the existing two-model teaching loop on Ascend 910C.

• Use your own prompts with Qwen3.5-0.8B proposing and 2B verifying. Stream committed groups or cancel at a commit boundary.
• Sample with positive temperature and top-k/top-p. Each model keeps its actual transformed probabilities; verification and cache recovery reuse the existing loop.
• Start without checkpoints: run python -m minideepinfer.sampling_filters_demo in a compatible CPU PyTorch environment.
• Install a fixed version: both setup guides now use v0.5.0, with wheel, source archive and SHA256SUMS attached below. No PyPI upload.

Runtime modules are unchanged from f40f6af; this release updates metadata and documentation. Installed-package checks run outside the checkout. The test matrix has 251 passing tests with Transformers 5.12.1, 239 passing and 12 skipped with 4.46.3, and 79 passing and 172 skipped without model dependencies.

Existing NPU records remain tied to their original commits: temperature sampling passes 28 probability / 6 cold-state checks including actual draft recovery; filtered sampling passes 24 / 5 with all 3/3 proposals accepted. Filtered rejection is covered on CPU. Probabilities and filtering stay on CPU FP64, with FP32 model execution on Ascend 910C. No new NPU run for packaging and no performance or speedup claim.

中文

这一版把双模型教学接到自己的文本：逐组看输出，在提交边界取消，也可以用温度、top-k / top-p 改变采样范围。两模型使用各自实际过滤后的概率，候选验证、回退和缓存回收仍走同一条同步循环。

先按安装指南准备兼容 CPU PyTorch，再运行 python -m minideepinfer.sampling_filters_demo，无需权重或 NPU。中英文安装指南统一到固定 tag；旧版本、原始结果和历史提交保持不变。

本轮只整理版本与文档，运行模块与 f40f6af 相同，复用已有实机证据，不重新加载模型。附件提供 wheel、源码包与校验和，未上传 PyPI。模型和角色素材遵循各自的权利说明。

中文安装指南 · English setup · 完整版本说明 / Release notes · top-k / top-p 教程

v0.4.0 — Follow draft proposals through verification and recovery

v0.4 follows a draft proposal through verification, correction, cache recovery and the next round. The draft and target each keep their own cache, using the existing synchronous greedy verification path.

With compatible CPU PyTorch, run the installed package without checkpoints:

python -m minideepinfer.model_draft_demo --cpu-example rejection --trace-rounds --width 3 --output outputs/draft-rejection.json

The untrained random models produce rejected candidates without answer lookup or candidate injection. Watch the draft move from six computed inputs back to four valid ones, then consume the correction to reach five. Omit --cpu-example rejection for an all-accepted comparison.

• Round observations show actual candidates, model inputs, commits, logical cache lengths and cleanup. They add no model calls and run separately from timing.
• Tokenizer compatibility is checked before real weights load. Width-1 draft snapshots are skipped when no tentative candidate is fed back.
• Failed drafts cannot be reused. Interruptions and cleanup errors preserve failure reports, including unknown target cleanup as NOT_OBSERVED.
• All 214 tests pass with Transformers 5.12.1; 202 pass and twelve skip with 4.46.3; the base environment passes 75 and skips 139. Both CPU walkthroughs match their preserved reports when installed outside the checkout.
• The Ascend 910C FP32 release check uses Qwen3.5-0.8B → 2B: widths 1/3 accept 4/5 and 6/7, match independent greedy output, pass sixteen cold-cache comparisons and return all fourteen slots per model. Each existing checkpoint loads once; the full program makes fifty forward calls. No new performance timing or speedup claim.

This is a teaching implementation, separate from cross-round generation. It does not implement MTP/DSpark, probability-preserving sampling or SWA Bounded Replay. Earlier slower and small-effect timing results retain their original source revisions. Old releases and raw reports are preserved.

Assets: wheel, source distribution and SHA256SUMS. This release is not uploaded to PyPI.

中文

这一版把“两模型猜错后怎样继续”串成可运行的逐轮演示：候选来自小模型实际预测，打印 target 的验证与提交，以及下一轮 draft 恢复前缀的过程。CPU 不需要权重；真实 0.8B/2B 已完成 Ascend 910C 正常生成与状态验收。异常后禁止复用半份缓存，失败报告保留中断、清理错误和未知状态。

逐轮说明与状态图 · 完整发布说明 · 实现与历史实验

v0.3.0 — Greedy verification and Qwen3.5 state recovery

这一版讲清候选验证之后的状态处理：哪些输出可以提交，哪些 KV 还能读，Qwen3.5 的卷积窗口和递推矩阵怎样恢复。

三个 CPU 入口按顺序运行，无需下载权重：

python -m minideepinfer.spec_demo
python -m minideepinfer.spec_demo --hybrid
python -m minideepinfer.spec_demo --hybrid --draft lookup

先用受控候选观察拒绝与 bonus，再检查混合状态的快照和重算，最后让 lookup 只根据已提交历史提出候选。独立候选模式在生成完成后才运行 greedy 对照。

Ascend 910C 上的真实 Qwen3.5-2B FP32 输出和冷缓存状态检查均通过。lookup 短例子接受 0/5，target 工作量由 10 forward / 14 输入 token 增至 12 / 21；没有计时或提速结论。这个结果和快照、查找、重算成本一起保留。

189 项测试在 Transformers 5.12.1 下全部通过；4.46.3 为177通过、12跳过；基础环境为70通过、119模型检查跳过。三个 CPU 模式均通过安装后的仓库外检查。本次发布整理不改执行源码，复用已有实机报告，没有重新加载模型。

安装方法、预期输出和六份报告的固定版本见完整说明。附件包含 wheel、源码包和 SHA256SUMS；尚未发布到 PyPI。

English

v0.3.0 follows a candidate through greedy verification, output commit, KV rollback, and Qwen3.5 state recovery. The three CPU commands above need PyTorch but no checkpoint. The lookup draft reads committed history only; its independent greedy baseline runs afterwards.

Preserved real Qwen3.5-2B FP32 checks on Ascend 910C match greedy output and cold-cache state. Lookup accepted none of five proposals and increased target work from 10 forwards / 14 input tokens to 12 / 21. There is no timing or speedup claim. Snapshot, lookup, replay and reference costs remain separate. This is a synchronous teaching path; it does not implement learned drafts, MTP/DSpark, probability-preserving sampling, SWA Bounded Replay or cross-round speculation.

All 189 tests pass with Transformers 5.12.1, and all three installed CPU modes run outside the checkout. Release preparation reuses preserved hardware evidence without changing execution source or reloading weights. See the release walkthrough for setup, scope and evidence. Assets include a wheel, source distribution and SHA256SUMS; this release is not on PyPI.

v0.2.0 — Follow three requests through the pipeline

v0.2 把跨轮执行接到真实文本生成，并增加 A、B、C 的完整调度演示：A 的长 prompt 分块计算，B 先结束，C 排队后复用 B 的缓存槽位。

已有兼容 PyTorch 时，无需下载权重即可运行：

python -m minideepinfer.pipeline_demo

中文讲解 · English walkthrough · 运行指南 / Setup

真实文本使用 generate --pipeline；--stagger-prompts 在收取结果时逐条加入后续文本，--trace-rounds 打印轮次与缓存归属。调度继续复用原有循环，没有新增引擎。

150 项测试在 Transformers 5.12.1 环境全部通过，安装包已在仓库目录外检查。Ascend 910C 上沿用 Qwen3.5-2B FP32 的 43 组完整 logits 对照：最大绝对差约 4.94e-5，输出一致，缓存全部归还；十八份相关执行源码 SHA 保持一致。本次演示和发布整理没有重新加载真实权重，也没有性能提速结论。

English

v0.2 connects the existing cross-round pipeline to text generation. The new A/B/C walkthrough runs a tiny random CPU model: A has a chunked prompt, B finishes first, and C waits before reusing B's slots. Final outputs are checked against whole-prompt synchronous execution with independent caches.

Use generate --pipeline for real text, --stagger-prompts for callback-driven arrivals, and --trace-rounds for observations. Cross-round text execution remains single-device, eager, greedy, FP32, and intended for short teaching examples.

All 150 tests pass with Transformers 5.12.1. Installed-wheel checks run outside the checkout. Preserved Ascend 910C / Qwen3.5-2B results pass 43 full-logit comparisons at the existing tolerances. The related execution source is unchanged; no additional hardware run or performance claim is made for this release.

Wheel, source distribution, and SHA256SUMS are attached. The existing v0.1.0 tag and assets are preserved. See the full release notes for scope and validation details.

mini-DeepInfer v0.1.0

mini-DeepInfer v0.1 可以在 Ascend 910C 上运行 Qwen3.5-2B 文本生成。项目参考 DeepInfer 的设计，把输入准备、模型计算、采样和结果提交拆成可运行的小实验，便于跟着请求读代码。

没有 NPU 也可以开始：基础例子只用 Python，随后换成随机初始化的 PyTorch 小模型，最后再运行真实权重。十四课依次讲到批次调度、两种缓存、设备 token 接续，以及请求取消后怎样等设备用完再回收。

• 中文安装指南 / English setup
• 十四课索引 / 完整发布说明
• CI / 实机报告与源码

发布前补了不同输入长度、预算、到达和取消时刻的调度对照。140 项测试在 Transformers 5.12.1 下全部通过；wheel 安装后也在仓库目录之外运行过基础例子和随机模型检查。

Qwen3.5-2B 的 Ascend 910C FP32 实测包含十四组完整 logits 对照，最大绝对差约 2.67e-5，在原容差内。四个中间 prompt 块不采样，结束后归还 24/24 槽位。报告中的十五份相关源码 SHA 与本版本一致。

主例使用单设备、eager、greedy 和 FP32，适合短上下文教学。跨轮调度仍通过独立实验运行，尚未接入 generate；BF16 在不同扫描和调度下的生成差异也已记录。现有少量短请求计时没有证明稳定提速。

附件提供 wheel、带教程和报告的源码包，以及 SHA256 校验值。可从本页下载，尚未上传 PyPI。

English

mini-DeepInfer v0.1 runs text-only Qwen3.5-2B on Ascend 910C and uses small experiments to explain DeepInfer's execution flow. Start with Python-only examples, move to tiny PyTorch models, then inspect the same ideas with real weights.

The release includes batching, prompt chunks, split launch/commit execution, and a separate cross-round experiment for arrivals, stops, and cache reuse. All 140 tests pass with Transformers 5.12.1. The saved FP32 device run passes fourteen full-logit comparisons, with a maximum absolute difference of about 2.67e-5 and all 24 slots reclaimed. Its source hashes match this version.

This is a single-device teaching project. Cross-round scheduling is separate from generate, BF16 differences remain documented, and timing so far has not shown a consistent speedup.

Independent community project, with no official affiliation with DeepSeek, SGLang, or Huawei. Original code and documentation use Apache-2.0; model and third-party licenses remain applicable.