Validation Gate

An evidence-based completion gate for Codex skills and other agent workflows. It turns a requirements checklist into a script-generated verification report, so an agent cannot legitimately declare a deliverable complete merely because it says so.

中文说明

What it does

validation-gate separates intent from proof:

1. Define requirements with stable IDs in requirements.json.
2. Run deterministic generic checks and optional artifact-specific adapters.
3. Write verification.json from code.
4. Allow completion only when every required check is pass.

Required checks use three states:

• pass — a validator produced evidence.
• fail — a validator found a violation.
• unknown — no trustworthy automated proof is available.

unknown never permits completion.

Included checks

The bundled verifier supports safe, deterministic checks:

• path_exists
• file_nonempty
• text_contains
• regex_matches
• json_path_exists
• json_path_equals
• adapter_result

Use an adapter for facts that require inspecting a concrete artifact, such as a PPTX object, a rendered PDF, DOCX section structure, or paper citation coverage. See adapter authoring.

Quick start

Create requirements.json:

{
  "requirements": [
    {
      "id": "output-created",
      "description": "The deliverable exists and is non-empty.",
      "checks": [
        { "kind": "path_exists", "path": "output/deliverable.txt" },
        { "kind": "file_nonempty", "path": "output/deliverable.txt" }
      ]
    }
  ]
}

Run the gate:

python3 scripts/verify_contract.py \
  --contract requirements.json \
  --output verification.json

The command exits with 0 only when all required checks pass. Otherwise it writes the report and exits with 1.

Install as a Codex skill

Copy this directory into your Codex skills directory:

cp -R validation-gate ~/.codex/skills/

Start a new Codex task, then invoke it with $validation-gate or ask Codex to use an evidence-based validation gate.

Trust model

This skill prevents accidental hallucinated completion claims. It is not a security boundary if the same agent can freely change the artifact, verifier, and report. For compliance-sensitive workflows, run the verification step in protected CI or an external service with read-only access to the artifact.

中文说明

validation-gate 是一个面向 Codex skill 和 agent 工作流的“完成门禁”。它将需求清单转换为脚本生成的验证报告，避免模型只因自己写了“已完成”就宣称交付成功。

工作方式

1. 在 requirements.json 中为每条需求定义稳定 ID。
2. 运行确定性的通用检查，必要时接入特定产物的 adapter。
3. 由代码写入 verification.json。
4. 只有所有必需检查均为 pass 时，才允许模型宣布完成。

状态含义：

• pass：验证器生成了证据。
• fail：验证器发现不符合要求。
• unknown：当前没有可信的自动化验证方式。

unknown 不允许宣布完成。

适用场景

• PPT：检查文件存在、对象 ID、图标资源、溢出和渲染结果。
• 论文：检查章节、字数、引用覆盖率、参考文献、图表标题和占位符。
• DOCX/PDF：检查结构、文本、页数、格式及渲染结果。
• 代码与数据：检查文件、测试结果、JSON 字段和规则匹配。

通用验证器只处理可确定的文件和数据事实。对 PPT 对象、PDF 渲染、论文引用覆盖率等内容，使用 adapter 检查实际产物；参考 adapter 编写说明。

快速开始

创建 requirements.json 后运行：

python3 scripts/verify_contract.py \
  --contract requirements.json \
  --output verification.json

只有全部必需项通过，命令才会以 0 退出；否则会写出报告并以 1 退出。

安装到 Codex

将本目录复制到 ~/.codex/skills/，然后新开一个 Codex 任务。你可以用 $validation-gate 显式调用，或要求 Codex 使用“基于证据的验证门禁”。

信任边界

它适合防止模型无意中幻觉式地声明完成；但如果同一个 agent 同时拥有修改产物、验证器和报告的权限，它并不是安全边界。对合规或对抗性场景，应将验证步骤放在受保护的 CI 或外部只读服务中执行。