Proposal: task-conditioned Harness evolution with evaluable Plugin Packs

[wangcan26]

Summary

DSH can already inspect and change its own runtime through Cordis, and
developers can distribute those changes as plugins. The current specification
does not bind the developer's task, exact active composition, execution trace,
evaluation, and outcome into reusable evidence. The platform can distribute
plugin artifacts, but it cannot systematically accumulate evidence about how
they perform across tasks, models, and environments or use that evidence to
update Harness Recipes, Scorecards, and composition policies. This is the
missing link in the evolution of externalized Harness knowledge.

I propose a task-conditioned Harness evolution loop. The platform treats every
Plugin and Harness Recipe as a versioned data asset. Each asset declares a Task
Profile that defines its task family and applicability boundary, then ships a
representative evaluation suite. The platform runs those evaluations, maintains
independent Scorecards, and gives the AI evidence for searching the next
composition. Real task outcomes update that evidence.

Here, the platform is a logical registry and evaluation role. A local CI
system, an organization service, a community registry, or an official hosted
service could perform that role. It stores each trial result separately and
builds a Scorecard only from results produced under comparable versions and
conditions.

The proposed asset contract binds four records that are currently separate:
the exact artifact, its task claims and evaluation cases, platform-produced
scores, and the outcomes observed when a Harness used it.

task
  -> retrieve compatible plugins and compositions
  -> search composition trajectories
  -> probe candidates against the current task and retention anchors
  -> issue a Selection Receipt and guarded CommitDecision
  -> activate an accepted harness for this task
  -> execute the task and record the outcome
  -> update shared evidence and future search priors

I will use Harness Pack as a provisional name for the publisher-owned
contract that attaches task claims, reproducible evaluation cases, graders,
and compatibility metadata to either a Plugin Asset or a Harness Recipe.
Platform-produced results, Scorecards, and real-task Outcomes remain separate
records. DSH already uses Package for an immutable version of a dynamic Cordis
Plugin, so the final name should avoid that collision.

Related community foundations

Several community projects provide useful foundations:

• #1597
  and #1760
  provide natural-language plugin discovery.
• #2282
  defines use_when / not_for capability declarations, recommends Top-K
  plugins before a task, and publishes a small B/C evaluation with scripts.
• #1888
  proposes task-driven, session-scoped plugin assembly and cleanup.
• #2276
  proposes machine discovery, trust metadata, and a shareable Compose unit
  containing plugin references and security declarations. Its comments also
  discuss runtime evidence for both plugins and compositions.
• #2269
  connects marketplace records to machine-readable verification reports through
  fields such as verifiedBy, verifiedAt, and reportUrl.
• #2088
  proposes real-host release lifecycle validation for packed plugins.
• #524
  examines retrieval and staged activation for large tool catalogs.

These discussions solve different boundaries. Discovery finds candidates.
Capability metadata explains when an author expects a plugin to help. Validation
checks whether an artifact installs and behaves as declared. Compose records a
reusable configuration. Session assembly applies a configuration to a task.

This proposal adds a platform-owned contract for task evidence, composition
search, outcome attribution, and Recipe promotion. It turns the discovery,
capability metadata, validation, Compose, and session-assembly work above into
one task-conditioned Harness evolution loop.

What DSH provides today, and what is missing

DSH already has several prerequisites for an evolvable harness. They establish
the mechanics of change and observation, but they do not decide which change
will improve the current task.

Current mechanism	What it establishes	Missing condition for self-evolution
Cordis composition and the dynamic package lifecycle	DSH can inspect, add, run, stop, replace, and dispose components without rebuilding the agent loop.	No task objective, candidate generator, search policy, or reward function chooses a composition. The model-facing dynamic package toolset is also opt-in and process-local.
Skill Registry	DSH can merge local, embedded, remote, and runtime skill catalogs and load a selected skill.	The registry has no standard task profile, evaluation suite, Scorecard, or evidence-based ranking for combinations.
Plugin inventory and inspection	DSH can report what is installed or running.	Inventory is an observation of the current runtime, not a searchable history of candidate recipes and their task outcomes.
Session Log and reconstructable request headers	DSH can preserve the exact model-visible history, tool calls, and changing tool catalog for a run.	The log does not bind a task outcome to an evaluated Harness Recipe or attribute the result to a composition change.
Feedback and session telemetry	A deployment can capture user feedback and export redacted execution records.	No evaluator consumes those records, updates a Scorecard, or changes the next composition search. Telemetry handoff alone does not create learning.
Plan Mode	DSH can inject planning guidance and require human review before execution.	Current Plan Mode is soft prompt guidance. It has no world model, reward, candidate tree, rollout, or automatic selection.

The current architecture therefore provides mutation, composition, and
observation. Harness self-evolution also requires search, evaluation,
selection, and retention.

Here, retention means more than keeping a trace. A successful combination must
become a versioned asset with reproducible evidence, then influence the search
for later tasks.

Upgrade 1: make task evidence part of the asset model

The platform could manage two related assets:

• A Plugin Asset is one installable capability.
• A Harness Recipe is an immutable, versioned artifact with a content
  fingerprint. It records the exact plugins, model settings, prompts, skills,
  and configuration needed to rebuild one Harness composition.

The second asset matters because plugin effects are not additive. Two useful
plugins can conflict, duplicate context, increase tool-selection errors, or
exceed a task's latency and token budget. DSH therefore needs evidence for both
individual plugins and complete harness combinations.

The core evaluation contract connects each asset to two records. The planner
also reads mutable Selection Facts, described in Upgrade 2.

1. Publisher-owned Pack

The immutable Pack contains declarations and reproducible evaluation inputs:

• task profile, including the task family, applicability boundaries, and
  expected outcomes;
• compatibility and permission requirements;
• representative evaluation suites and graders;
• exact plugin or recipe versions;
• known conflicts and declared side effects.

2. Platform-maintained evidence

The platform runs evaluations and stores the resulting Scorecard separately:

• exact Pack, model, environment, dataset, and grader versions;
• trial count and outcome distribution;
• quality, cost, latency, and safety metrics;
• comparison with the active baseline;
• retention anchors for historical tasks and protected capabilities;
• provenance and links to reproducible reports.

The publisher may contribute evaluation cases and graders. A publisher-supplied
score: 95 should not become trusted platform evidence.

One successful trial should not broaden an asset's claimed applicability. The
Pack states the intended task family. Platform evidence determines where that
claim holds, while retention anchors protect earlier behavior when a Recipe
changes.

This split extends the evidence fields discussed in #2269. It also gives the
Compose work in #2276 a task-quality dimension rather than treating a
composition as useful solely because it can be shared and activated.

A possible Pack shape

pack:
  id: code-review-security
  version: 0.3.0
  kind: plugin # plugin | harness-recipe
  publisher: example-org

  artifact:
    plugins:
      - id: security-review
        version: 1.4.2

  task_profile:
    intents:
      - review a code change for security vulnerabilities
    inputs:
      - git_diff
      - repository_context
    expected_outcomes:
      - actionable findings with file and line evidence

  compatibility:
    models:
      - family: deepseek
    environments:
      - git_repository
    permissions:
      - filesystem_read
    conflicts: []

  evaluation:
    suites:
      - id: security-review-v1
        cases: ./evals/cases.jsonl
        graders: ./evals/graders.yaml
    required_metrics:
      - finding_precision
      - finding_recall
      - task_success
      - token_cost
      - latency

The existing Compose schema from #2276 may be the right base for
harness-recipe; a separate Harness Pack type may not be necessary. The point
is the contract between the asset, its task claims, and independently produced
evidence.

Upgrade 2: build a task-conditioned Harness planner

The planner searches over Harness configurations instead of returning one
static recommendation:

• Its state contains the current task, active Harness Recipe, environment, and
  partial execution evidence.
• Its actions add, remove, replace, or configure Plugin Assets.
• Its objective combines task quality, cost, latency, safety, and user
  feedback.
• Its output is a versioned Harness Recipe supported by evidence for the
  current constraints.

Artifact identity alone is not enough for selection. Compatibility,
reachability, setup requirements, permissions, cost, risk, and evidence
references can change without changing the Plugin or Recipe itself. The
platform should store these as separately versioned Selection Facts. Each
choice should emit a Selection Receipt that pins the facts, constraints,
and evidence used for that decision.

An independent runner should handle exact-artifact execution binding as a
separate interoperability contract. Before it produces evaluation evidence,
the runner should resolve a versioned RunnerBinding. That document binds
the Selection Receipt to the selected candidate identifier, exact artifact
digest, artifact byte source, and named digest profile. The digest profile must
define portable JSON canonicalization and artifact byte-source rules, with
implementation-independent conformance vectors. The runner should place a
required runnerBindingRef in EvaluationEvidence so the evidence digest
commits to the binding. The evaluator or gate must reject unresolved,
invalid, or mismatched bindings before CommitDecision.

The Pack and evidence contracts should not prescribe one search algorithm. The
platform can begin with retrieval, constraint filtering, ranked candidate
generation, and a small number of probes. Later implementations may use beam
search, a contextual bandit, tree search, or a trained planning model.

Current DSH Plan Mode does not perform this search. It injects planning guidance
and collects human approval for one generated plan.

Upgrade 3: close the task-to-evidence loop

For a request, DSH or a platform service could:

1. Classify the user's goal and current harness context against one or more Task
   Profiles.
2. Retrieve compatible Plugin Assets and Harness Recipes.
3. Resolve current Selection Facts, then filter candidates by model,
   environment, permissions, cost, risk, reachability, and conflicts.
4. Generate composition actions: keep, add, remove, replace, or reconfigure.
5. Rank candidate trajectories with evaluation evidence from similar tasks.
6. Run isolated task-specific probes when the evidence is weak or stale.
7. Select a candidate Recipe and emit a Selection Receipt that binds the exact
   facts, constraints, and evidence behind the choice.
8. An independent runner materializes the selected candidate and resolves a
   versioned RunnerBinding that binds the receipt to the exact artifact and
   named digest profile.
9. Evaluate that exact candidate in a controlled runtime against the current
   task, retention anchors, validity checks, and budget limits. Produce
   EvaluationEvidence with a required runnerBindingRef.
10. Emit a CommitDecision bound to the candidate, evidence, and policy digests.
    Activate the candidate through Cordis only when the decision accepts it.
11. Execute the real task and bind its trace, outcome, cost, and user feedback
    to that Recipe.
12. Update the Scorecard, applicability boundary, and retention evidence.
13. Register a successful composition as a new versioned Harness Recipe after
    it passes reproducibility, safety, and provenance checks.

The planner selects a better-supported harness under the current task, model,
environment, budget, and risk constraints.

The activation step can build on #1888. Candidate retrieval can build on #1597,
#1760, and #2282. The Pack and evidence records can build on #2276, #2269, and
#2088. Cordis supplies the mutation actions. The proposed addition is the
planner and the evaluation-driven feedback contract between those parts.

The platform should split work between two loops:

• Offline evidence loop: validate Packs, run repeatable suites, compare
  recipes against both current-task suites and retention anchors, and publish
  versioned Scorecards. This gives online search useful priors.
• Online task loop: search only a bounded compatible candidate set, run
  cheap probes where needed, emit a Selection Receipt, commit only an accepted
  candidate, and feed the observed outcome back into platform evidence.

Without the offline loop, every developer pays to rediscover the same
combinations. Without the online loop, the platform can only recommend a static
average that may not fit the current task.

Why this fits DSH

Cordis already supplies the execution substrate: composition, inspection,
versioned dynamic packages, activation, update, teardown, and effect disposal.
The Skill Registry supplies a discovery surface. Session logs and telemetry can
provide execution evidence.

DSH already supports controlled mutation. The proposed layers add the missing
learning loop:

Pack contract
  -> platform evaluation and Scorecard
  -> task-conditioned planner
  -> Selection Receipt
  -> isolated probe and retention checks
  -> CommitDecision
  -> Cordis activation
  -> real task outcome
  -> evidence update and Recipe promotion

With this loop, DSH can use past outcomes to improve future composition
decisions.

Future direction: train DeepSeek for Harness search

The platform loop also creates training data:

Task Profile
  + starting Harness Recipe
  + available Plugin Assets
  + Selection Facts and constraints
  + candidate composition actions
  + Selection Receipt
  + RunnerBinding
  + current-task and retention evidence
  + CommitDecision
  + final task outcome

DeepSeek could use these records to train a model specialized in Harness search
and planning. Given a task and the current recipe, the model could predict the
next composition action, estimate candidate value, or decide that the current
recipe is sufficient. Successful search traces provide supervised targets;
platform evaluation and real outcomes can provide reward or preference signals.

The model would accelerate candidate generation and reduce online probes. It
would not replace the Pack contract or external evaluation: the model proposes
and predicts, while the platform verifies and records the result. This is a
future direction, not a capability DSH currently provides.

Related research

• The Cordis paper describes the temporal and spatial composition mechanisms
  needed to replace components safely:
  https://github.com/cordiverse/paper
• AlphaEvolve combines candidate generation, automated evaluators, a program
  database, and selection of promising candidates:
  https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
• Anthropic's agent-evaluation guidance separates tasks, trials, graders,
  traces, and outcomes, and treats the evaluated agent as its model and harness:
  https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
• Voyager stores verified executable skills and retrieves them for later tasks:
  https://arxiv.org/abs/2305.16291
• RAP shows that a planner can search reasoning trajectories with task rewards
  while balancing exploration and exploitation. It helps explain the problem;
  this proposal does not adopt RAP or require MCTS:
  https://arxiv.org/abs/2305.14992
• Harness Continual Learning separates candidate generation from guarded state
  commitment and checks current improvement, historical retention, and
  validity before committing a harness update:
  https://arxiv.org/abs/2608.19013

Open questions

1. Should task and evaluation metadata extend the Compose work in [RFC] Ecosystem primitives: plugin discoverability, an optional identity seam, and a path to account-anchored sync #2276, extend
   the plugin manifest, or use a separate Pack contract?
2. Which evaluation inputs belong in the immutable asset, and which results
   belong in platform-maintained evidence?
3. What is the smallest useful first scope: one plugin, one complete Harness
   Recipe, or both?
4. Which graders can the platform run reproducibly across different plugin
   categories?
5. How should DSH compare results across models and environments without hiding
   the differences behind one aggregate score?
6. How should the platform handle private datasets, benchmark leakage,
   malicious graders, and score gaming?
7. Which parts should remain portable across harnesses, and which parts should
   describe Cordis-specific lifecycle rules?
8. Which search policy should the first planner use after retrieval and
   constraint filtering?
9. Where should candidate probes run so that failed or malicious combinations
   cannot mutate the user's active Harness?
10. Which records may be used to train a Harness planning model, and how should
    the platform preserve user and organization privacy?

Would it be useful to define a minimal shared schema that existing discovery,
Compose, verification, and session-assembly projects can all produce or consume?

If you would like the DSH team to evaluate these contracts and runtime hooks,
please upvote this Discussion. Use the comments for implementation evidence,
objections, and design feedback.

[wangcan26]

概要

DSH 已经可以通过 Cordis 检查并改变自身运行时，也可以通过插件包分享这些改变。
但当前规范没有把开发者面临的任务、实际启用的组合、执行 Trace、评测与 Outcome
绑定为可复用证据。平台因此可以分发插件，却不能系统积累插件在不同任务、模型和
环境中的有效性经验，也无法据此更新 Harness Recipe、Scorecard 和组合策略。
这正是 Harness 外化知识进化所缺失的环节。

这份提案引入面向任务的 Harness 自进化循环。平台把每个 Plugin 和 Harness Recipe
作为带版本的数据资产。每项资产通过 Task Profile 声明适用的任务族和边界，并携带
有代表性的评测套件。平台负责运行评测、维护独立 Scorecard，并为 AI 搜索下一组
插件组合提供证据。真实任务 Outcome 继续更新这些证据。

这里的“平台”是 Registry 与评测服务的逻辑角色，可以由本地 CI、组织服务、社区
Registry 或官方托管服务承担。平台分别保存每次 Trial 的结果，只聚合相同版本与
条件下的结果来生成 Scorecard。

这项资产契约把四类记录绑定在一起：精确产物、任务声明与评测 Case、平台生成的评分，
以及 Harness 实际使用该资产后产生的 Outcome。

任务
  -> 检索兼容的插件与组合
  -> 搜索 Harness 组合轨迹
  -> 针对当前任务和 Retention Anchor 运行 Probe
  -> 生成 Selection Receipt 与受保护的 CommitDecision
  -> 激活当前任务下通过检查的 Harness
  -> 执行真实任务并记录 Outcome
  -> 更新共享证据和后续搜索先验

下文暂用 Harness Pack 指代发布者维护的契约：它为 Plugin Asset 或 Harness
Recipe 附加任务声明、可复现评测 Case、Grader 与兼容性元数据。平台生成的评测结果、
Scorecard 与真实任务 Outcome 作为独立记录保存，不写入 Pack。DSH 已经用 Package
表示动态 Cordis Plugin 的不可变版本，最终名称应当避免混淆。

社区相关基础

以下 Discussions 提供了可复用的基础：

• #1597
  和 #1760
  提供自然语言插件发现。
• #2282
  定义了 use_when / not_for 能力声明，在任务开始前推荐 Top-K 插件，并发布了
  一组小规模 B/C 评测和评分脚本。
• #1888
  提出由任务驱动的会话级插件装配与清理。
• #2276
  提出机器发现、信任元数据，以及包含插件引用和安全声明的可分享 Compose 单元。
  该帖评论还讨论了插件与组合的运行时证据。
• #2269
  用 verifiedBy、verifiedAt、reportUrl 等字段，把市场条目连接到机器可读的
  验证报告。
• #2088
  提出对插件实际打包产物执行真实宿主发布生命周期验证。
• #524
  讨论了大规模工具目录的检索与分阶段激活。

这些讨论分别解决了不同边界。发现机制负责找出候选插件。能力元数据表达作者认为
插件适用于什么场景。验证机制检查产物能否安装，以及行为是否符合声明。Compose
保存可复用配置。会话级装配将配置应用到当前任务。

本提案新增平台维护的任务证据、组合搜索、结果归因与 Recipe 提升契约，并将上述
插件发现、能力声明、运行验证、Compose 与会话装配串成一条面向任务的 Harness
自进化链路。

DSH 目前提供了什么，还缺什么

DSH 已具备 Harness 演化的多项前提。它们解决了“如何改变”和“如何记录”，但没有
回答“什么改变能改善当前任务”。

当前机制	已经具备的能力	自进化仍缺少的条件
Cordis 组合与动态 Package 生命周期	DSH 无需重建 Agent Loop，就能检查、添加、运行、停止、替换和释放组件。	没有任务目标、候选生成器、搜索策略和奖励函数负责选择组合。面向模型的动态 Package 工具还需要显式启用，状态只存在于当前进程。
Skill Registry	DSH 可以合并本地、内嵌、远程和运行时 Skill 目录，并加载选中的 Skill。	Registry 没有统一的 Task Profile、评测套件、Scorecard，也不能根据证据比较组合。
Plugin Inventory 与运行时检查	DSH 可以报告已经安装或正在运行的组件。	Inventory 只是当前运行时快照，不是包含候选 Recipe 及其任务结果的可搜索历史。
Session Log 与可重建的请求头	DSH 可以保存模型可见历史、工具调用，以及变化后的工具目录。	Session Log 没有把 Task Outcome 绑定到经过评测的 Harness Recipe，也不能归因某次组合变化。
Feedback 与 Session Telemetry	部署方可以采集用户反馈，并导出经过脱敏的执行记录。	没有 Evaluator 消费记录、更新 Scorecard，或者改变下一次组合搜索。遥测上报本身不会产生学习。
Plan Mode	DSH 可以注入规划指导，并在执行前要求用户评审。	当前 Plan Mode 是 Prompt 软约束，没有世界模型、奖励函数、候选树、Rollout 和自动选择。

因此，当前架构已经提供变异、组合和观察。Harness 自进化还需要搜索、评测、
选择和沉淀。

这里的沉淀不只是保留 Trace。成功组合需要成为带版本、可复现的资产，并影响后续
任务的搜索。

升级一：把任务证据纳入资产模型

平台可以管理两类相关资产：

• Plugin Asset 是一个可安装能力。
• Harness Recipe 是一份不可变、带版本和内容指纹的组合产物，记录重建一套
  Harness 所需的精确插件、模型设置、Prompt、Skill 和配置。

Harness Recipe 不能省略。插件效果并不满足简单加法。两个单独有用的插件组合后，
仍可能互相冲突、重复占用上下文、增加工具误选，或者超出任务的时延与 Token 预算。
平台既要保存单插件证据，也要保存完整 Harness 组合的证据。

核心评测契约把每项资产连接到两类记录。Planner 还会读取独立维护的 Selection Facts，
详见“升级二”。

1. 发布者维护的 Pack

不可变 Pack 保存声明和可复现的评测输入：

• Task Profile，包括任务族、适用边界和预期 Outcome；
• 兼容性和权限要求；
• 有代表性的评测套件与 Grader；
• 精确的插件或 Recipe 版本；
• 已知冲突和声明的副作用。

2. 平台维护的证据

平台运行评测，并单独保存 Scorecard：

• 精确的 Pack、模型、环境、Dataset 和 Grader 版本；
• Trial 数量和 Outcome 分布；
• 质量、成本、时延和安全指标；
• 与当前基线的比较；
• 历史任务与受保护能力的 Retention Anchor；
• 来源以及可复现报告链接。

发布者可以贡献评测 Case 和 Grader，但发布者自报的 score: 95 不应直接成为平台
可信证据。

一次成功 Trial 不能扩大资产声明的适用范围。Pack 声明目标任务族，平台证据确定
这一声明在哪些条件下成立；Recipe 发生变化时，Retention Anchor 用于保护既有行为。

这个拆分扩展了 #2269 讨论的证据字段，也为 #2276 的 Compose 增加任务质量维度。
一个组合能够分享和激活，只能证明它是可用资产；评测结果才能说明它适合哪些任务。

一种可能的 Pack 结构

pack:
  id: code-review-security
  version: 0.3.0
  kind: plugin # plugin | harness-recipe
  publisher: example-org

  artifact:
    plugins:
      - id: security-review
        version: 1.4.2

  task_profile:
    intents:
      - review a code change for security vulnerabilities
    inputs:
      - git_diff
      - repository_context
    expected_outcomes:
      - actionable findings with file and line evidence

  compatibility:
    models:
      - family: deepseek
    environments:
      - git_repository
    permissions:
      - filesystem_read
    conflicts: []

  evaluation:
    suites:
      - id: security-review-v1
        cases: ./evals/cases.jsonl
        graders: ./evals/graders.yaml
    required_metrics:
      - finding_precision
      - finding_recall
      - task_success
      - token_cost
      - latency

#2276 的 Compose Schema 可能已经适合承载 harness-recipe，未必需要新增 Harness Pack
类型。提案的重点是建立资产、任务声明与独立评测证据之间的契约。

升级二：建设面向任务的 Harness Planner

Planner 搜索 Harness 配置，而不是返回一个静态推荐：

• State 包含当前任务、活跃 Harness Recipe、运行环境与部分执行证据。
• Action 包括添加、移除、替换或配置 Plugin Asset。
• 目标函数综合任务质量、成本、时延、安全性与用户反馈。
• 输出是一个带版本、且有当前任务证据支持的 Harness Recipe。

仅靠 Artifact 身份不足以完成选择。兼容性、可达性、配置要求、权限、成本、风险和
证据引用都可能在 Plugin 或 Recipe 不变时发生变化。平台应把这些信息保存为独立
版本的 Selection Facts。每次选择都应生成 Selection Receipt，固定当时
使用的事实、约束和证据。

独立 Runner 应把精确产物执行绑定作为单独的互操作契约。生成评测证据前，Runner
应解析带版本的 RunnerBinding，将 Selection Receipt 与选中的 Candidate ID、
精确 Artifact Digest、Artifact Byte Source 和具名 Digest Profile 绑定。Digest
Profile 必须定义可移植的 JSON 规范化方式和 Artifact Byte Source 规则，并提供与具体
实现语言无关的一致性测试向量。Runner 应在 EvaluationEvidence 中写入必需的
runnerBindingRef，使 Evidence Digest 提交该绑定。Evaluator 或 Gate 必须在
CommitDecision 前拒绝无法解析、校验失败或不匹配的绑定。

Pack 与证据契约不应绑定某一种搜索算法。平台可以先实现检索、约束过滤、候选排序和
少量 Probe。后续实现可以采用 Beam Search、Contextual Bandit、Tree Search 或
经过训练的规划模型。

当前 DSH Plan Mode 没有执行这类搜索。它注入规划指导，并让用户评审模型生成的一份
Plan。

升级三：闭合任务与证据循环

对于一次请求，DSH 或平台服务可以执行以下流程：

1. 根据一个或多个 Task Profile 对用户目标和当前 Harness 上下文进行归类。
2. 检索兼容的 Plugin Asset 和 Harness Recipe。
3. 获取当前 Selection Facts，再按模型、环境、权限、成本、风险、可达性和冲突关系
   过滤候选项。
4. 生成组合动作：保留、添加、移除、替换或重新配置。
5. 使用相似任务的评测证据对候选轨迹排序。
6. 当证据不足或已经过期时，在隔离环境运行面向当前任务的小型 Probe Case。
7. 选择候选 Recipe，并生成 Selection Receipt，绑定此次选择使用的精确事实、约束和
   证据。
8. 独立 Runner 物化选中的 Candidate，并解析带版本的 RunnerBinding，将 Receipt
   与精确 Artifact 和具名 Digest Profile 绑定。
9. 在受控运行时中，用当前任务、Retention Anchor、有效性检查和预算限制评测该精确
   Candidate，并生成包含必需 runnerBindingRef 的 EvaluationEvidence。
10. 生成绑定 Candidate、Evidence 和 Policy Digest 的 CommitDecision。只有
    CommitDecision 接受后，Cordis 才激活该 Candidate。
11. 执行真实任务，将 Trace、Outcome、成本和用户反馈绑定到该 Recipe。
12. 更新 Scorecard、适用边界和 Retention Evidence。
13. 当成功组合通过可复现性、安全性和来源检查后，将其登记为新版 Harness Recipe。

Planner 在当前任务、模型、环境、预算和风险约束下，选择证据更充分的 Harness。

激活环节可以建立在 #1888 之上。候选检索可以复用 #1597、#1760 和 #2282。
Pack 与证据记录可以沿用 #2276、#2269 和 #2088 的成果。这份提案新增的是这些
环节之间的 Planner，以及由评测驱动的选择与反馈契约。

平台需要分开运行两个循环：

• **离线证据循环：**验证 Pack、针对当前任务套件与 Retention Anchor 运行可复现
  评测、比较 Recipe，并发布带版本的 Scorecard，为在线搜索提供先验。
• **在线任务循环：**只搜索一组有界且兼容的候选项，必要时运行低成本 Probe，
  生成 Selection Receipt，只提交通过 CommitDecision 的 Candidate，并把真实
  Outcome 回流到平台证据。

缺少离线循环时，每个开发者都要重复发现相同组合。缺少在线循环时，平台只能推荐
一个静态平均结果，未必适合当前任务。

为什么适合 DSH

Cordis 已经提供执行基础：组合、检查、带版本的动态 Package、激活、更新、卸载和
副作用清理。Skill Registry 提供发现入口。Session Log 与 Telemetry 可以提供
执行证据。

DSH 已经支持受控变异。提案增加缺失的学习循环：

Pack 契约
  -> 平台评测与 Scorecard
  -> 面向任务的 Planner
  -> Selection Receipt
  -> 隔离 Probe 与 Retention 检查
  -> CommitDecision
  -> Cordis 激活
  -> 真实任务 Outcome
  -> 更新证据并提升 Recipe

完成这条链路后，DSH 可以根据过去的 Outcome 改善后续组合决策。

未来方向：训练擅长 Harness 搜索的 DeepSeek 模型

平台循环会产生以下训练数据：

Task Profile
  + 初始 Harness Recipe
  + 可用 Plugin Asset
  + Selection Facts 与约束
  + 候选组合动作
  + Selection Receipt
  + RunnerBinding
  + 当前任务与 Retention Evidence
  + CommitDecision
  + 最终任务 Outcome

DeepSeek 可以利用这些记录训练专门负责 Harness 搜索与规划的模型。模型接收当前任务
与 Recipe 后，可以预测下一步组合动作、估计候选价值，或者判断当前 Recipe 已经足够。
成功搜索轨迹可以提供监督信号，平台评测和真实 Outcome 可以提供 Reward 或偏好信号。

该模型可以减少候选生成和在线 Probe 的成本，但不能替代 Pack 契约与外部评测。模型
负责提出候选和预测结果，平台负责验证并记录结果。这是未来方向，不是 DSH 当前已经
具备的能力。

相关研究

• Cordis 论文讨论了组件安全替换所需的时间可组合性与空间可组合性：
  https://github.com/cordiverse/paper
• AlphaEvolve 把候选生成、自动 Evaluator、程序数据库和候选选择放进同一个循环：
  https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
• Anthropic 的 Agent Eval 文章区分了 Task、Trial、Grader、Trace 和 Outcome，
  并把模型与 Harness 共同作为评测对象：
  https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
• Voyager 保存经过验证的可执行 Skill，并在后续任务中检索使用：
  https://arxiv.org/abs/2305.16291
• RAP 说明 Planner 可以利用任务奖励搜索推理轨迹，并在探索与利用之间取得平衡。
  它用于解释问题，本提案不采用 RAP，也不要求 MCTS：
  https://arxiv.org/abs/2305.14992
• Harness Continual Learning 将 Candidate 生成与受保护的状态提交分开，并在提交
  Harness 更新前检查当前任务增益、历史能力保留和有效性：
  https://arxiv.org/abs/2608.19013

希望讨论的问题

1. Task 和 Evaluation 元数据应该扩展 [RFC] Ecosystem primitives: plugin discoverability, an optional identity seam, and a path to account-anchored sync #2276 的 Compose、扩展 Plugin Manifest，
   还是使用独立 Pack 契约？
2. 哪些评测输入应写入不可变资产，哪些评测结果应由平台单独维护？
3. 第一版的最小范围应该是单插件、完整 Harness Recipe，还是同时支持两者？
4. 哪些 Grader 可以跨插件类型由平台稳定复现？
5. DSH 应该如何比较不同模型和环境下的结果，避免用一个聚合分数掩盖重要差异？
6. 平台应该如何处理私有 Dataset、Benchmark 泄漏、恶意 Grader 和刷分？
7. 哪些契约应该跨 Harness 复用，哪些内容应该专门描述 Cordis 生命周期？
8. 第一版 Planner 在检索和约束过滤后，应该采用哪种搜索策略？
9. 候选 Probe 应该在哪里运行，才能防止失败或恶意组合改变用户正在使用的 Harness？
10. 哪些记录可以用于训练 Harness 规划模型，平台应如何保护用户和组织的隐私？

是否值得先定义一份最小公共 Schema，让现有的发现、Compose、验证和会话装配项目
都能生产或消费同一份任务与评测证据？

如果你希望 DSH 团队评估这些契约与运行时 Hook，请为本 Discussion 点 Upvote。
实现证据、异议和设计建议请继续留在评论区。

[WayneJin0918]

非官方补充，贴一个已经能跑的窄域 Pack 草图。

#2454 把缺的环节写清楚了：任务、组合、评测、Scorecard 现在是分开的。世界模型研究刚好是一个很小的落地：

• task profile：late-horizon / action-following / 认错路线（把 Sora、Gaussian、Dreamer 写成一套配方）
• eval case：一个 rollout 目录 + 可选 wm.yaml（pred / gt / log）
• grader：纯 JS luminance SSIM/MSE，再强制看 worst frames（contact sheet）
• RSI 对象：skill / wm.yaml / eval notes（Harness 层），不碰训练权重

社区插件 dsh-wm 按这条回路做了 measure / know / iterate。sunset fixture 不占 GPU，适合当最小 probe。

这回答不了你列出的平台级问题（跨模型 Scorecard、恶意 grader、隐私），但可以当「一个 plugin Pack + 一个可复现 eval case」的最小范围。

相关：Show Your Plugins #2618

[wangcan26]

Wayne，感谢你把 dsh-wm 和 #2454 的 Pack 设想对应起来。我看了仓库，它已经具备
一个窄域 Pack 的关键部分：明确的任务场景、可复现 Fixture、Grader、Harness Skill，
以及修改后的回归门禁。

我想邀请 dsh-wm 成为这份提案的第一个社区 Reference Pack / Conformance Fixture
候选。这里的 Reference 指用真实插件检验公共契约，不代表 DSH 官方认证。

我把完整路线拆成四个阶段，避免把一次联合实验说成完整的 Harness 自进化：

阶段	要验证的问题	需要建设的能力	本次范围
M1 Pack Evidence	单个 Pack 能否被独立、可复现地评测？	pack.yaml、统一 Eval 命令、隔离 Runner、evidence.json、Artifact/Dataset/Grader 摘要	完整验证
M2 Recipe Scorecard	哪个精确 Recipe 在固定任务条件下更好？	recipe.lock、基线与候选 Recipe、多次 Trial、指标聚合和可追溯 Scorecard	最小验证
M3 Task Planner	AI 能否根据当前任务选择更合适的组合？	候选 Registry、约束过滤、Planner、Probe、精确激活、回滚和审计	本次不做
M4 Evolution Loop	历史 Outcome 能否改善后续选择？	Outcome 回流、结果归因、Scorecard 更新、Recipe 提升、回滚和隐私治理	本次不做

这次可以把联合实验控制在以下范围。

dsh-wm

• 增加最小 pack.yaml，声明 Artifact、Task Profile、Sunset Eval、Grader 和兼容性。
• 世界模型专属字段放在 extensions.world_model，不写入通用 Schema。
• 提供稳定的结构化评测入口，例如：

dsh-wm eval --suite sunset --json

• 保留 Sunset 作为 Grader 回归 Fixture，再增加一个 Agent 级任务，验证 Agent 是否
  能识别 late-horizon collapse，并引用 SSIM 和 worst frames。

契约与独立验证

我可以先起草通用的 pack.yaml、evidence.json 和 recipe.lock 最小 Schema，
再建立一个独立 Runner：

• 固定原始 DSH 与 DSH + dsh-wm 两个 Recipe；
• 在相同模型和环境下运行多次 Trial；
• 由 Runner 实时计算结果，不采用 Pack 内预填分数；
• 生成第一份本地 Scorecard，并保留到每次 Trial 的追溯链接。

这次实验的终点是：完整验证 M1，并用两个固定 Recipe 验证 M2 的最小证据与
Scorecard 契约。M3 的组合搜索和 M4 的 Outcome 学习留到后续，不在 dsh-wm
这次接入中实现。

这个范围对你合适吗？如果可以，我先把最小 Schema 草案贴到 #2454，再根据 dsh-wm
的实际结构调整，最后通过一个小 PR 验证契约。

[Jstn-1g]

@WayneJin0918 @wangcan26，我把上面讨论的最小 evaluator→evidence→decision 接缝实际跑通并固定下来了：

• runner 会先验证 dsh-wm@cd06866a638b1dcc80defa6d674ff3470ff0ad31、Sunset pred/GT trees、package.json 与 scorer blobs，并拒绝相关路径中的修改或未跟踪文件；
• 实时 scorer 结果为 mean SSIM 0.734452382311、first-half 0.993063755288、second-half 0.475841009333、worst frames [4,5,6]；
• 两个明确标注为 synthetic oracle 的 conformance 场景中，GT→pred regression 被拒绝，pred→GT recovery 被接受；
• 同一个 prediction 在 baseline/candidate 角色互换时保持同一 artifact digest，最终 decision 同时绑定 policy 与 evidence digest。

实现与限制说明：
https://github.com/Jstn-1g/dsh-guarded-hcl/tree/v0.1.0/integrations/dsh-wm

独立 conformance CI：
https://github.com/Jstn-1g/dsh-guarded-hcl/actions/runs/32546943177

v0.1.0：
https://github.com/Jstn-1g/dsh-guarded-hcl/releases/tag/v0.1.0

这里仍然只是 M1 scorer-to-contract conformance，不是两套真实 Recipe 的 M2 对比，也不是 live Cordis adapter；oracle regression 的 current-task mean 本身也下降，因此我没有把它包装成“当前任务提升但 retention 下降”的实验结果。

我把真正的 baseline/candidate run pair 拆成了一个可认领的下一步：
Jstn-1g/dsh-guarded-hcl#1

如果 dsh-wm 后续提供稳定的 JSON CLI（例如讨论里的 dsh-wm eval --suite sunset --json），adapter 就不必依赖内部 rolloutDiff export。你们更倾向让真实 run manifest 与 evidence.json mapping 放在 dsh-wm、这个 gate repo，还是独立 runner 中？

[Jstn-1g]

@WayneJin0918 I went ahead and implemented the smallest upstream seam as a short PR: WayneJin0918/dsh-wm#1

It adds optional diff --json by serializing the existing rolloutDiff result, preserves the current human-readable output by default, and adds a public-CLI regression covering the half-horizon metrics plus bilingual docs and changelog notes.

The ownership boundary stays narrow: dsh-wm exposes domain evaluation output; checkout provenance, runtime context, run manifest, and EvaluationEvidence mapping remain runner-owned; promotion policy remains gate-owned.

Verification: Linux/Node 22 tests pass 25/25, npm run check passes, the installed-bin JSON path parses successfully, and default human output matches cd06866a byte-for-byte. I left a new versioned envelope out of this first PR so the change can expose the existing result without prematurely freezing a broader schema. Happy to adjust the shape or naming during review.

[wangcan26]

@Jstn-1g The public CLI seam in
dsh-wm PR #1 matches the
ownership split we agreed on. dsh-wm exposes its existing domain result,
while checkout provenance, runtime context, run manifests, and
EvaluationEvidence remain runner-owned. Keeping the default human output
unchanged and leaving the larger versioned envelope out of this PR also keeps
the first change narrow.

I would treat the PR as a candidate M1 evaluator interface until Wayne reviews
and merges its final shape. The stable boundary should preserve a few
properties:

• JSON mode writes one machine-parseable result to stdout and sends diagnostics
  to stderr;
• field names, numeric types, warnings, and failure exit codes remain
  deterministic;
• the evaluator exposes enough scorer identity for the runner to record the
  exact implementation it invoked;
• the default human-readable output remains covered by regression tests.

The independent runner should continue to compute checkout, dataset, ground
truth, prediction, and runtime identities. Those fields do not need to move
into dsh-wm merely to make the first CLI seam useful.

Once the upstream shape is accepted, dsh-guarded-hcl can replace its temporary
internal rolloutDiff import with the public CLI and pin the merged revision.
That improves M1 conformance. The real baseline/candidate run pair in
dsh-guarded-hcl issue #1
remains the separate M2 experiment.

[Jstn-1g]

The newly released Harness Continual Learning paper (arXiv:2608.19013) appears to formalize an important part of the evolution loop described here: separating candidate generation from state commitment.

A paper-faithful mapping to DSH could use three independently replaceable Cordis boundaries:

1. ContinualOptimizer: receives a frozen harness snapshot plus post-execution feedback and emits a versioned candidate affecting the task interface, experience memory, capability map, or adaptive router.
2. ContinualEvaluator: evaluates the candidate against the current task, protected historical-retention anchors, and code-owned validity checks under a controlled runtime.
3. CommitController: activates the exact candidate only when the evidence passes configured improvement, retention, validity, and budget thresholds; otherwise the active harness remains unchanged.

The important distinction is that rollback after a failure is not identical to guarded commitment. The candidate should ideally remain isolated until the evaluator authorizes promotion, and the optimizer should not own its own rubric or aggregate decision.

Would it be useful to define a small shared contract for Candidate, EvaluationEvidence, RetentionAnchor, and CommitDecision so existing community optimizers and continual-memory plugins can interoperate instead of each inventing a different scorecard and promotion rule?

I have drafted a small, dependency-free reference gate and JSON Schemas that demonstrate deterministic accept/reject decisions, per-anchor non-regression checks, validity gates, evidence digests, and a deliberately regressing fixture. I would be glad to align the names and fields with the Pack / Recipe / Scorecard terminology being developed in this discussion.

This is an independent, early contract draft and executable reference gate—not an official DeepSeek project, not yet a live Cordis plugin, and not a security sandbox. It does not silently mutate cross-session state: the optimizer only proposes; an independent evaluator and code-owned commit controller decide whether to promote the exact candidate.

Reference implementation: https://github.com/Jstn-1g/dsh-guarded-hcl

[wangcan26]

Thanks for turning this boundary into an executable gate and for keeping the
evidence claims narrow.

I agree that guarded commitment belongs between candidate evaluation and
Cordis activation. I updated the proposal to make three points explicit:

1. A Task Profile describes a task family and applicability boundary, while
   retention anchors protect historical tasks and capabilities.
2. The platform evaluates the exact candidate against current-task
   improvement, retention, validity, and budget constraints before activation.
3. A CommitDecision binds the candidate, evidence, and policy digests. Cordis
   activates only an accepted candidate.

A small shared contract for Candidate, EvaluationEvidence,
RetentionAnchor, and CommitDecision looks useful. I would keep ownership
strict:

• Candidate references the exact baseline and Recipe/content digest.
• An independent runner produces EvaluationEvidence.
• The platform or deployment policy owns RetentionAnchor and thresholds.
• A code-owned gate produces the deterministic CommitDecision.

On the run-manifest question from the dsh-wm thread, my preference is:

• dsh-wm owns the stable domain evaluation interface and its Pack declarations;
• an independent runner owns checkout verification, runtime context, the run
  manifest, and the mapping into evidence.json;
• the gate consumes normalized evidence and does not depend on dsh-wm internal
  exports.

That keeps publisher declarations, observed evidence, and the commit decision
under separate ownership. I would describe dsh-guarded-hcl as a Reference
Gate and M1 conformance fixture today. It does not need to claim M2 Recipe
comparison or a live Cordis adapter to validate this boundary.

The HCL paper supports the proposal/commit separation. The three Cordis
components remain one possible DSH mapping rather than a requirement of the
portable contract.

[Jstn-1g]

@wangcan26 Thank you—this resolves the ownership question cleanly. I’ll keep dsh-guarded-hcl scoped and labeled as a Reference Gate + M1 conformance fixture.

I’ll treat the current in-repo dsh-wm adapter as a temporary pinned seam and target this split:

1. dsh-wm owns a public, stable JSON evaluation interface and its Pack declarations.
2. An independent runner owns checkout verification, runtime context, the run manifest, and mapping into EvaluationEvidence.
3. The gate consumes only normalized evidence and has no dependency on dsh-wm internal exports.
4. A future Cordis adapter consumes only an accepted CommitDecision; M2 Recipe comparison remains a separate experiment.

@WayneJin0918, the only upstream seam needed for the next step is stable evaluator output. Would a minimal envelope like this fit dsh-wm?

schemaVersion, suite, scorerRevision,
datasetDigest, groundTruthDigest, predictionDigest,
pairCount, metrics, worstFrameIndexes, warnings

For Sunset, metrics would expose at least mean_ssim, min_ssim, first_half_ssim, and second_half_ssim. If that boundary looks right, I’ll draft the independent run-manifest → evidence.json mapping in Jstn-1g/dsh-guarded-hcl#1 before implementing against it.

[YE-YI7]

A narrow executable data point from a separate selection-layer experiment may help with the contract split in this proposal.

I have a public Draft PR that packages ASM as a native DSH bundle: YE-YI7/asm-spec#13

It contributes two tools: asm_list_services first exposes the bounded catalog, then asm_select chooses only within one required taxonomy and emits an unsigned Selection Receipt. The tarball was installed and booted against DSH 0.1.1-rc.2 in a clean profile; the local smoke listed two travel candidates and preserved critical risk + approval-required in the receipt.

Important limitation: this adapter currently selects services in an ASM catalog, not arbitrary installed DSH plugins. It is not a planner, installer, evaluator, execution gate, or adoption claim. The implementation deliberately requires an explicit taxonomy because the current selector does not treat natural-language task text as a semantic classifier; an unbounded cross-category result would look plausible while being invalid. approval_required is also not authorization.

The boundary that emerged maps to the Pack discussion as four separate objects:

1. immutable plugin/recipe artifact identity and reproducible eval inputs;
2. platform-owned observed scorecards/outcomes;
3. mutable selection facts: compatibility, reach, setup requirements, permissions, cost, risk, and evidence references, pinned by their own digest;
4. a per-decision Selection Receipt recording which exact facts and constraints produced the choice. Receipt v0.1 stays explicitly unsigned.

I would avoid putting mutable scores or pricing into the artifact version identity. A manifest-only change should change the selection-evidence digest without creating a new plugin artifact version.

For the smallest common schema, would DSH prefer a namespaced URI + digest for these selection facts in package/Bundle metadata, or a separate Pack/registry sidecar? I do not want to overload ToolSchema, freeze a community-only package field, or duplicate the evaluator/commit-controller contracts. If the direction is useful, I can reduce the Draft PR to a DSH-plugin fixture that proves only this identity-versus-selection-digest boundary.

[Jstn-1g]

@YE-YI7 I checked PR #13 at 5b6dd8c from the guarded-commit side.

Within its stated synthetic, representation-level scope, yes: it demonstrates the intended identity/selection split. The Bundle digest stays stable, the Selection Facts digest changes, and the deterministic receipt pins the taxonomy, constraints, full two-candidate set, and both facts digests.

I would keep the namespaced Bundle-metadata reference exactly as you label it: a provisional fixture encoding, not a claim that DSH has accepted that field placement or a live sidecar-resolution contract.

The next interoperability seam is exact-artifact binding. The current receipt binds service_id + facts_digest, while the Bundle artifact digest is recorded separately. An independent runner manifest could bind { service_id, bundleDigest, selectionReceiptDigest } before producing EvaluationEvidence. That gives a clean chain:

ASM Selection Facts -> Selection Receipt
independent runner -> exact Bundle + receipt -> EvaluationEvidence
dsh-guarded-hcl -> evidence + policy/retention -> CommitDecision
future Cordis adapter -> activate only an accepted exact candidate

This preserves your stated boundary: selection risk and approval_required are facts, not execution authorization, and the unsigned receipt remains an audit record. Strong narrow M3 conformance fixture; thank you for making the distinction executable.

[YE-YI7]

@Jstn-1g Thank you for checking the exact fixture. I agree that exact-artifact binding is the next seam, but I do not think it should be folded into the current PR #13 fixture.

I found two contract blockers that should be resolved before calling the next step interoperable:

1. dsh-guarded-hcl EvaluationEvidence v0.1 has additionalProperties=false and no artifact-binding or runner-binding field. An ASM-only runner manifest would therefore be recorded outside the evidenceDigest and could not be verified by the current CommitDecision chain.
2. ASM currently hashes Python canonical JSON while guarded-hcl hashes a recursively sorted JavaScript JSON.stringify result. They agree for this integer-only fixture, but JSON number forms such as 3.0 versus 3 can diverge across runtimes. We need one named canonicalization rule or shared cross-language digest vectors before binding a Python receipt digest into Node evidence.

I updated PR #13 at 8467ae9 to state both limits explicitly and replaced hard-coded fixture assertions with computed checks. CI is green: Python, registry, and DSH plugin jobs all pass.

My proposed sequencing is:

• keep PR 好棒 #13 as the narrow Selection Facts / Selection Receipt seam;
• define exact-artifact binding as a separate interop contract;
• add either an artifactBinding object to a future EvaluationEvidence version, or a required runnerBindingDigest that points to a small runner manifest;
• pin digest scope and canonicalization with Python/Node shared test vectors;
• then test that changing the Bundle, receipt, service_id, or binding digest fails before EvaluationEvidence reaches CommitDecision.

I lean toward runnerBindingDigest so the HCL evidence contract does not have to absorb ASM-specific fields, but only if the referenced runner-manifest schema and digest construction are versioned and mandatory. Would that fit your intended evidence boundary, or would you prefer a first-class artifactBinding object in EvaluationEvidence?

[wangcan26]

@YE-YI7 I checked PR #13 at 8ac1a8f, including the Windows determinism fix.
The Python, registry, DSH plugin, and Windows jobs are green, and the latest
independent re-review approved the fix. Within the stated synthetic scope, the
fixture now demonstrates the Selection Facts / Selection Receipt boundary
without claiming a live resolver or planner.

I agree with keeping exact-artifact binding out of PR #13. The two blockers you
identified require a separate interoperability contract:

1. EvaluationEvidence v0.1 cannot commit to an informal external manifest
   because its schema is closed.
2. A cross-runtime digest needs a named canonicalization rule and shared test
   vectors. Artifact identity also needs a declared byte source, path policy,
   line-ending policy, and other metadata rules that fall inside its digest
   scope.

For the next seam, I prefer the runnerBindingRef direction proposed in
dsh-guarded-hcl issue #5.
EvaluationEvidence v0.2 should require a schema URI plus digest for a versioned
RunnerBinding. The referenced document should own:

• the candidate or service identifier;
• the exact Bundle or artifact digest and its digest profile;
• the artifact byte source;
• the Selection Receipt schema URI and digest.

This keeps ASM-specific placement out of the gate while ensuring the evidence
digest commits to the binding. The runner or evaluator should fail before
CommitDecision when it cannot resolve the reference, validate the schema, or
match any bound digest.

This does not add a Python dependency to DSH. The reference runner can remain
Node/TypeScript; the conformance vectors only verify that external producers
generate portable digests.

RFC 8785 JCS with SHA-256 is a reasonable first JSON digest profile, but the
contract still needs implementation-independent vectors before we freeze it.
The vectors should cover number forms, Unicode ordering, duplicate keys,
invalid numbers, and artifact-byte cases such as LF/CRLF, nested paths,
symlinks, and packaged versus working-tree identity.

That gives us a clean sequence:

Selection Facts / Selection Receipt fixture
  -> separate RunnerBinding contract
  -> EvaluationEvidence v0.2 runnerBindingRef
  -> CommitDecision

PR #13 can remain the narrow M3 representation fixture. RunnerBinding/JCS then
becomes the next independent M3-to-M1/M4 interop seam.

[YE-YI7]

@wangcan26 Thank you — I kept PR #13 narrow and opened the first separate digest-profile slice here: Jstn-1g/dsh-guarded-hcl#6

It adds a dependency-free Node implementation of RFC 8785 JCS + SHA-256 plus implementation-neutral vectors containing canonical text, exact UTF-8 bytes, digests, and rejection results. The vectors cover recursive data, the RFC UTF-16 ordering case, 3.0/3, -0, exponent boundaries, subnormal values, the largest finite double, escaped-equivalent duplicate keys, invalid numeric tokens, and lone surrogates. Local verification is 30/30; fork PR workflows currently await maintainer approval.

The scope remains intentionally smaller than RunnerBinding: no EvaluationEvidence v0.2, no artifact/source-tree digest, and no claim that JSON canonicalization solves LF/CRLF, paths, modes, symlinks, or packaged-vs-working-tree identity. After the JSON profile is reviewed, the artifact byte-source profile and typed binding/evidence seam can be separate follow-ups.

[Jstn-1g]

@YE-YI7 Verified and merged: Jstn-1g/dsh-guarded-hcl#6

Both required PR checks and all post-merge main-branch checks (CI, dsh-wm conformance, and CodeQL) pass. I also reproduced the 30/30 test result, verified the npm package contents, and matched canonical output against all six files in the RFC 8785 reference corpus.

This merge accepts only the bounded JCS + SHA-256 digest-profile slice. Issue #5 remains open for the artifact byte-source profile, RunnerBinding, EvaluationEvidence v0.2, cross-language execution, and fail-closed mutation tests.

Thank you for turning the proposed seam into an executable contribution.

[Jstn-1g]

@YE-YI7 Yes—this sequencing fits the intended evidence boundary. I checked 8467ae9; the revised limitations and computed checks now bound PR #13 accurately, and I do not think exact-artifact binding should be folded into that fixture.

One precision: EvaluationEvidence v0.1 already carries generic baselineDigest and candidateDigest fields. What it lacks is a typed Selection Receipt → exact artifact binding under a declared digest profile. Because v0.1 is a closed schema, that reference should arrive in a new evidence version rather than as an informal v0.1 extension.

My preference is a required runnerBindingRef object in EvaluationEvidence v0.2:

{
  "schemaUri": "https://…/runner-binding-v0.1.schema.json",
  "digest": "sha256:…"
}

The referenced, versioned RunnerBinding would own the candidate/service identifier, exact Bundle/artifact digest and digest profile, plus the Selection Receipt schema URI and digest. That keeps ASM-specific placement out of the gate while ensuring evidenceDigest commits to the binding reference. Failure to resolve, validate, or match it should stop before CommitDecision. It remains provenance/audit binding—not authorization or a signature.

I also agree that “recursively sorted JSON” is not a portable contract. The digest profile should name one rule—my leading choice is RFC 8785 JCS with SHA-256 over its UTF-8 canonical bytes—and ship shared Python/Node vectors for canonical bytes, digests, and rejection cases. Those vectors should cover 3.0/3, -0, exponent boundaries, Unicode/non-BMP ordering, nested data, duplicate keys, non-finite numbers, and lone surrogates.

I opened the follow-up as a public, help-wanted contract issue with the mutation properties and definition of done: Jstn-1g/dsh-guarded-hcl#5

So: keep PR #13 narrow; define this as the next separate seam; then prove that mutations to the Bundle, receipt, service ID, binding schema, or binding digest fail before evidence reaches the gate.

[YE-YI7]

@wangcan26 Closure update on the exact M3-to-M1/M4 seam you described ("RunnerBinding/JCS then becomes the next independent M3-to-M1/M4 interop seam"):

• ASM PR 好棒 #13 is merged — the M3 Selection Receipt fixture you reviewed at head 8ac1a8f: Add bounded DeepSeek Harness selector plugin YE-YI7/asm-spec#13
• the generic contracts landed upstream in Jstn-1g/dsh-guarded-hcl (#11), and guarded-HCL issue #5 is now closed as completed: Define EvaluationEvidence v0.2 runner binding and portable JSON digests Jstn-1g/dsh-guarded-hcl#5
• ASM PR 求一个memory能力 #14 is now merged — merge commit 7b956ba, reviewed fixture head 7604868 — implementing the chain you outlined (Selection Receipt → RunnerBinding v0.1 → EvaluationEvidence v0.2 with required runnerBindingRef, failing closed before CommitDecision), with one positive chain plus 12 independent negative mutations and green ubuntu-latest/windows-latest runs: test: validate ASM receipt through guarded HCL evidence YE-YI7/asm-spec#14

Would you confirm whether this matches the seam you intended? A yes/no plus any remaining boundary correction is enough. I am not asking for an official DeepSeek adoption statement; the current claim remains contract_conformance only.

[wangcan26]

@YE-YI7 Yes — for the stated contract_conformance scope, this matches the seam I intended.

I independently verified:

• ASM PR #13 and
  PR #14 are merged at the
  commits cited above;

• guarded-HCL PR #11
  is merged and
  issue #5 is closed;

• the chain is explicit and fail-closed:

  Selection Receipt -> RunnerBinding v0.1 -> EvaluationEvidence v0.2 -> CommitDecision

• candidate identity, artifact kind and digest, artifact byte-source profile,
  Selection Receipt schema/profile/digest, and runnerBindingRef are bound;

• unresolved or independently mutated joins fail before CommitDecision;

• the final fixture has one positive chain plus 12 independent negative mutations,
  with the guarded-HCL consumer passing on Ubuntu and Windows.

The only scope precision I would retain is that this closes the
M3 selection-output to M1/M4 contract interoperability seam, not the
M1-M4 stages themselves.

It does not yet establish:

• an M1 Reference Pack with a public evaluator and independent task evidence;
• an M2 real Recipe comparison and Scorecard;
• an M3 task-conditioned planner, live Registry, Probe, or Bundle execution;
• an M4 Outcome feedback loop or Cordis activation/rollback;
• authorization, signatures, or official DeepSeek adoption.

Within that boundary, I have no remaining correction. I would treat this
interop seam as complete at the contract_conformance level. Thank you for
keeping both the implementation and the public claim precise.

[YE-YI7]

@wangcan26 Thank you for independently verifying the merged artifacts and for stating the scope precisely.

Recording the result as you stated: the Selection Receipt → RunnerBinding v0.1 → EvaluationEvidence v0.2 → CommitDecision chain matches the intended M3 selection-output → M1/M4 interoperability seam at contract_conformance scope, with no remaining correction. We do not claim completion of M1–M4, a live planner/evaluator/outcome loop, authorization, signatures, official DeepSeek adoption, or ecosystem adoption.

No further action needed from you on this seam. Any further claim would have to be earned separately — e.g., via a public evaluator with independent task evidence and a real Recipe comparison — rather than by extending this conformance result.

[tkornblit]

Yep, we're working on something similar here: https://tomkornblit.substack.com/p/we-gave-an-agent-write-access-to