Skip to content

v0.5.0 — Stream and sample with two models

Latest

Choose a tag to compare

@AkiYangDev AkiYangDev released this 25 Oct 09:58

v0.5.0 brings custom text, streamed output and probability sampling to the existing two-model teaching loop on Ascend 910C.

  • Use your own prompts with Qwen3.5-0.8B proposing and 2B verifying. Stream committed groups or cancel at a commit boundary.
  • Sample with positive temperature and top-k/top-p. Each model keeps its actual transformed probabilities; verification and cache recovery reuse the existing loop.
  • Start without checkpoints: run python -m minideepinfer.sampling_filters_demo in a compatible CPU PyTorch environment.
  • Install a fixed version: both setup guides now use v0.5.0, with wheel, source archive and SHA256SUMS attached below. No PyPI upload.

Runtime modules are unchanged from f40f6af; this release updates metadata and documentation. Installed-package checks run outside the checkout. The test matrix has 251 passing tests with Transformers 5.12.1, 239 passing and 12 skipped with 4.46.3, and 79 passing and 172 skipped without model dependencies.

Existing NPU records remain tied to their original commits: temperature sampling passes 28 probability / 6 cold-state checks including actual draft recovery; filtered sampling passes 24 / 5 with all 3/3 proposals accepted. Filtered rejection is covered on CPU. Probabilities and filtering stay on CPU FP64, with FP32 model execution on Ascend 910C. No new NPU run for packaging and no performance or speedup claim.

中文

这一版把双模型教学接到自己的文本:逐组看输出,在提交边界取消,也可以用温度、top-k / top-p 改变采样范围。两模型使用各自实际过滤后的概率,候选验证、回退和缓存回收仍走同一条同步循环。

先按安装指南准备兼容 CPU PyTorch,再运行 python -m minideepinfer.sampling_filters_demo,无需权重或 NPU。中英文安装指南统一到固定 tag;旧版本、原始结果和历史提交保持不变。

本轮只整理版本与文档,运行模块与 f40f6af 相同,复用已有实机证据,不重新加载模型。附件提供 wheel、源码包与校验和,未上传 PyPI。模型和角色素材遵循各自的权利说明。

中文安装指南 · English setup · 完整版本说明 / Release notes · top-k / top-p 教程