Skip to content

v0.2.0 — Follow three requests through the pipeline

Choose a tag to compare

@AkiYangDev AkiYangDev released this 24 Oct 09:43

v0.2 把跨轮执行接到真实文本生成,并增加 A、B、C 的完整调度演示:A 的长 prompt 分块计算,B 先结束,C 排队后复用 B 的缓存槽位。

已有兼容 PyTorch 时,无需下载权重即可运行:

python -m minideepinfer.pipeline_demo

中文讲解 · English walkthrough · 运行指南 / Setup

真实文本使用 generate --pipeline;--stagger-prompts 在收取结果时逐条加入后续文本,--trace-rounds 打印轮次与缓存归属。调度继续复用原有循环,没有新增引擎。

150 项测试在 Transformers 5.12.1 环境全部通过,安装包已在仓库目录外检查。Ascend 910C 上沿用 Qwen3.5-2B FP32 的 43 组完整 logits 对照:最大绝对差约 4.94e-5,输出一致,缓存全部归还;十八份相关执行源码 SHA 保持一致。本次演示和发布整理没有重新加载真实权重,也没有性能提速结论。

English

v0.2 connects the existing cross-round pipeline to text generation. The new A/B/C walkthrough runs a tiny random CPU model: A has a chunked prompt, B finishes first, and C waits before reusing B's slots. Final outputs are checked against whole-prompt synchronous execution with independent caches.

Use generate --pipeline for real text, --stagger-prompts for callback-driven arrivals, and --trace-rounds for observations. Cross-round text execution remains single-device, eager, greedy, FP32, and intended for short teaching examples.

All 150 tests pass with Transformers 5.12.1. Installed-wheel checks run outside the checkout. Preserved Ascend 910C / Qwen3.5-2B results pass 43 full-logit comparisons at the existing tolerances. The related execution source is unchanged; no additional hardware run or performance claim is made for this release.

Wheel, source distribution, and SHA256SUMS are attached. The existing v0.1.0 tag and assets are preserved. See the full release notes for scope and validation details.