DSH｜DSH Testkit｜真实宿主插件生命周期测试 / Real-host plugin lifecycle testing

[iiwish]

非官方项目，由社区成员独立开发和维护。

Unofficial project, independently developed and maintained by community members.

项目地址 / Project URL: https://github.com/iiwish/dsh-testkit
v0.3.1: https://github.com/iiwish/dsh-testkit/releases/tag/v0.3.1
npm: https://www.npmjs.com/package/dsh-testkit/v/0.3.1

中文

AI 可以很快写出一个 DSH 插件，但“代码能生成”“包能安装”并不能证明发布产物能在真实 DSH 中完成启动、注册、调用、卸载和重启。

DSH Testkit 是面向 DeepSeek Harness 插件的确定性真实宿主生命周期测试工具。它会把插件打包，在一次性 Docker 环境中安装精确版本的 DSH，启动真实宿主，验证配置与运行时注册，执行不依赖模型的最小能力调用，然后卸载插件并重启同一个 profile，最终保留 JSON、JUnit、Markdown、脱敏日志和阶段证据。

resolve → install-dsh → package → install-plugin → assemble → boot
        → register → exercise → update? → uninstall → reboot → recover? → cleanup

v0.3.1 一键接入

pnpm add -D dsh-testkit
pnpm dsh-test init
pnpm dsh-test

init 离线读取插件声明的 bundle patch，只生成三个可审核文件：

• dsh-testkit.yaml：精确 DSH 版本和自动识别的 row 预期
• .github/workflows/dsh-lifecycle.yml：最小权限的真实生命周期 CI
• .agents/skills/dsh-testkit/SKILL.md：让兼容的 coding agent 在开发、review 和发布时主动使用同一套门禁

命令幂等，不修改 package.json、lockfile 或 AGENTS.md；冲突或 symlink 路径会在任何目标写入前失败，除非显式使用 --force。

GitHub Actions 仍可直接使用：

- uses: iiwish/dsh-testkit/.github/actions/dsh-test@v0
  with:
    plugin: .
    dsh-version: 0.1.0-rc.6

它也提供可选的 DSH 原生 dsh_test tool 和 dsh-testkit Skill。外部 CLI/CI 仍是独立发布闸门：如果插件让宿主在 tool 注册前就启动失败，宿主内工具无法自救。

已验证的发布证据

• 111 个 unit、integration 和 contract tests
• 8 个真实 DSH 生命周期 fixture
• 真实 profile 中的原生 bundle、dsh_test 调用和 Skill discovery
• 干净公共 npm consumer、tarball scaffold 和完整卸载
• 受保护分支 CI、CodeQL、publint、actionlint、依赖审计和 npm SLSA provenance

它不做安全认证，也不评测模型输出质量。静态 manifest/preflight、plugin doctor、组合检查和插件单元测试仍然各有必要；Testkit 专注真实宿主中的单插件生命周期。

首批维护者招募

我们想邀请 5 个 DSH 插件维护者作为首批设计伙伴：

1. 以精确插件版本和精确 DSH 版本运行 Docker quick lifecycle。
2. 共同确认插件所需的 row/service/tool 与 prerequisite profile。
3. Testkit 维护者可以提交一个可审核的 CI 接入 PR。
4. 失败先私下或在插件仓库中提供可复现证据；未经复现和维护者知情，不做点名排名。
5. 反馈会优先影响 prerequisite profile、跨平台 runner 和下一批 Agent 工作流设计。

愿意参与可以直接在本帖留言，或在仓库创建 issue：
https://github.com/iiwish/dsh-testkit/issues

当前支持：@deepseek-ai/dsh@0.1.0-rc.6。DSH 仍处预览期，支持范围会以真实 canary 证据逐版本更新。

---

English

AI can generate a DSH plugin quickly, but generated code and a successful package install do not prove that the release artifact can boot, register, execute, uninstall, and reboot inside a real DeepSeek Harness host.

DSH Testkit packs the plugin, installs it beside an exact DSH version in a disposable Docker environment, boots the real host, verifies configuration and runtime registration, performs deterministic model-free exercise, uninstalls the plugin, reboots the same profile, and retains reviewable JSON, JUnit, Markdown, sanitized logs, and stage evidence.

v0.3.1 provides the three-command adoption flow above. dsh-test init works offline and creates only a scenario, least-privilege workflow, and project-local Agent Skill. It is idempotent, refuses implicit replacement and symlink paths, and does not edit package manifests, lockfiles, or repository instructions.

The same canonical dsh-testkit Skill ships in npm and registers in the native bundle when DSH exposes its optional Skills service. It helps compatible coding agents choose and interpret the lifecycle gate; it does not authorize untrusted execution or replace review.

A pass is environment- and scenario-bound lifecycle evidence. It is not a security certification, a model-quality evaluation, or a replacement for unit tests, static preflight, plugin doctors, or composition checks.

We are recruiting five DSH plugin maintainers as initial design partners. We will help define the smallest deterministic scenario, run the exact release in Docker, and prepare an opt-in CI integration PR. Findings are shared with reproducible evidence; named public failure claims are not made before reproduction and maintainer awareness.

Reply here or open an issue: https://github.com/iiwish/dsh-testkit/issues

Current host support: @deepseek-ai/dsh@0.1.0-rc.6.

[iiwish]

v0.3.0 已发布 / is released: https://github.com/iiwish/dsh-testkit/releases/tag/v0.3.0

一键接入新增 dsh-test init，会离线生成 scenario、最小权限 CI 和项目 Agent Skill；公共 npm 包、SLSA provenance、真实 DSH tool/Skill 注册均已验证。首批 5 位插件维护者招募继续开放，欢迎带一个精确插件版本来跑真实 lifecycle。

One-command setup now scaffolds the scenario, least-privilege CI, and project Agent Skill offline. The public npm package, SLSA provenance, and real DSH tool/Skill registration are verified. Recruitment remains open for the first five plugin maintainers.

[iiwish]

v0.3.1 已发布 / Released

我们把 v0.3.0 的一键接入实际应用到公开插件模板仓库根目录时，发现并修复了两个真实边界：Git source-resolution 日志没有进入严格 artifact 清单，以及带有合法 prepare 输出的 npm pack --json 不能只把整段 stdout 当作 JSON。

v0.3.1 已通过 Trusted Publishing 发布。公共 dsh-testkit@0.3.1 随后在模板仓库根目录完成了真实 DSH quick lifecycle：package、install、assemble、boot、register、deterministic exercise、uninstall、reboot 和 cleanup 全部通过。

• Release: https://github.com/iiwish/dsh-testkit/releases/tag/v0.3.1
• npm: https://www.npmjs.com/package/dsh-testkit/v/0.3.1
• Release CI: https://github.com/iiwish/dsh-testkit/actions/runs/31898835776
• Template integration PR: test: add real-host lifecycle gate bugmaker2/dsh-plugin-template#1
• Official Skill proposal: Proposal: add real-host release lifecycle validation to the plugin-development Skill / 建议增加真实宿主发布门禁 #2088

Applying the v0.3.0 onboarding to a public template repository root exposed two real boundaries: undeclared Git-resolution log artifacts and legitimate lifecycle output before the final npm pack --json result. v0.3.1 fixes both fail-closed, is public with provenance, and has passed the full quick lifecycle from the released package against the actual template root.

首批维护者招募继续开放。最有价值的下一步不是增加下载量，而是让不同形态的插件用精确版本跑出可复现证据。

[zoahdev]

This is the missing half of the release gate, and it pairs cleanly with the preflight half.

The division of labor I'd propose to plugin authors:

• dsh-plugin-doctor (mine) — fast, local, no-Docker preflight: manifest/patch validity, build, pack, fresh-profile install, --dump-config, entry-points, profile shadowing, BOM, runtime-dep and native-module tripwires. It's the "does this have a chance to boot" gate you run on every commit.
• dsh-testkit (yours) — the full real-host lifecycle: boot, register, deterministic exercise, uninstall, reboot, residue. The "does the packed artifact actually survive a real host end-to-end" gate for release.

Together they're the two halves of "generated code and a passing install are not proof it boots" — doctor catches the cheap failures before Docker, testkit catches the lifecycle failures after. The thing I'd emphasize to authors: run doctor in every CI run, run testkit on the release branch, and don't weaken either's expectations to make a pass.

Happy to be one of the five design partners — I maintain several plugins (dsh-subscribe, dsh-plugin-doctor, dsh-shelf, dsh-rule-evolve, dsh-pet-evolve) plus the dsh-plugin-template, so I can exercise the testkit across the "marketplace", "tool", "session CLI", and "self-evolution" plugin shapes and feed back what the prerequisite profile needs per shape. I'll also cross-link testkit from the doctor README so authors find both ends of the gate.

[iiwish]

Thank you. This is exactly the two-stage gate I hoped someone would make explicit. Counted as design partner #1.

I agree with the division and would keep the coupling loose:

• Doctor on every commit for fast, cheap preflight.
• Testkit on the release PR/tag for the exact packed artifact and real-host lifecycle.
• Both produce bounded evidence; neither is a security certification, and neither should be weakened to obtain a pass.

Template PR #1 already uses the public dsh-testkit@0.3.1 and keeps the runner outside the plugin dependency graph. I suggest using that as the baseline. Doctor can be added or cross-linked as the fast preflight without either project depending on the other.

For the first cohort run, dsh-plugin-doctor plus one behaviorally different plugin such as dsh-subscribe would give us two useful shapes. Reply with the exact releases you prefer; I will prepare the smallest declarative scenarios and report gaps as reproducible prerequisite-profile issues, never as a ranking. When the Doctor cross-link is live, I will add a reciprocal "fast preflight companion" link in the Testkit docs.

[leenkcool]

#1728 可以自己提PR收录 。记得互相赞赏。

[iiwish]

感谢提醒，也感谢你整理社区项目，已经点赞。

我们提交的 awesome-dsh-plugin PR #562 已于 2026-08-15 合并，DSH Testkit 现在已经进入该目录的中英文列表。

你在 #1728 介绍的 Blue-Whale-Harness 是另一份社区聚合目录。我刚确认它尚未收录 Testkit，并已经按仓库格式提交了 PR #10。我们会继续明确标注为社区项目，不把任何社区目录表述成官方背书。