feat(harness): agent milestone orchestrator skill, milestone-run profile, rolling canary cadence

[rickylabs]

Summary

The three artifacts of epic #1120, authored against the ratified design doc
(.llm/harness/design/milestone-orchestrator-and-canary-cadence.md, PR #1150): the
agent-milestone-orchestrator skill (the role), the milestone-run harness profile (the
run), and the rolling canary cadence (the schedule), wired to the release:canary-label
surface shipped by #1121/#1122. Draft PR per D1 — this PR is the ratification instrument.
Adversarial eval of record: review_claude (Codex · GPT-5.6 Sol · xhigh), owner-directed as the
merge gate; cycle 1 (9C/6M) fixed in d1a3e21c0; cycle 2 fixes + gate demonstrations in the
follow-up slice; two items escalated to the owner per the two-failure eval rule.

Scope

• Archetype / area: harness doctrine — .agents/skills/, .llm/harness/workflow/, generated
  .claude/skills/ mirror. No packages//plugins/ source.
• Closes epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120
• Refs verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 — the 0.0.6 verification issue owning the observational criteria (verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090 pattern)

Slices

• S0 run-dir bootstrap + locked outline — 7e66e6d64
• S1 workflow/canary-cadence.md — the schedule — 1774f6c95
• S2 workflow/milestone-run.md — the run — 04efa4b0e
• S3 agent-milestone-orchestrator skill + generated mirror + surface validation — 5268abe77
• S4 verification issue verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 (0.0.6, verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090 pattern) + acceptance evidence
• S5 Sol eval cycle-1 fixes C1–C9, M1–M6 — d1a3e21c0

Definition of Done

• Skill exists and encodes the pattern with cited evidence (run-from-it proof relocated to
  verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 — see the box note on epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120)
• Profile exists, distinct from the skill, no duplicated paragraph between them
• Cadence documented with trigger points tied to merge order, wired to release:canary-label
  (no reimplementation; canonical derivation is the release-canary.yml wiring)
• Every [observed] rule cites its recorded source (trace, filed issue, or ratified design
  record); no assertion promoted into an earned rule
• Every gate specified states its firing evidence honestly and a did-not-run signature; the
  two rows initially lacking a demonstrated negative case now have one (run-dir
  gate-demos.md: check-3 synthetic RED/GREEN demo; ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142 selection rule recovering the true
  verdict from merged PR ci: scope every expensive job to a classifier capability vector (#1152) #1155's live rollup)
• No publish mechanics restated from netscript-release; no second routing table
• .claude/skills/ regenerated via deno task agentic:sync-claude;
  deno task agentic:check-claude green
• Observational acceptance criteria routed to verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 (0.0.6)
• Adversarial eval findings triaged with per-finding dispositions; re-review on the same
  thread

Acceptance evidence

Entry text follows the close-gate mirror's contract: the verbatim first line of each issue
checkbox, with the final em-dash separating the evidence.

• agent-milestone-orchestrator skill exists and a fresh orchestrator can run a milestone — the skill exists and encodes the pattern with cited evidence (.agents/skills/agent-milestone-orchestrator/SKILL.md, commits 5268abe77 + d1a3e21c0); the run-from-it half is observational, relocated to verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 per the verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090 pattern (see the box annotation on epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120)
• A milestone/release harness profile exists, distinct from the skill, with no duplicated — .llm/harness/workflow/milestone-run.md (04efa4b0e); Sol cycles 2–3 ran exact-paragraph comparison across the three artifacts: zero duplicates
• Canary cadence is documented, with defined trigger points tied to merge order — .llm/harness/workflow/canary-cadence.md (1774f6c95): trigger = wave boundary [observed], membership computed from first-parent merge history between canary points
• Canary destination is visible in GitHub on issues and PRs using canary:<version>-canary.<n> — live since feat(release): derive canary labels from published versions #1122: canary:0.0.4-canary.1 applied on the payload (e.g. issue feat(release): canary label surface — derive the label from the published version, and make the payload content-derived #1121); canonical derivation passes the publish step's own output (release-canary.yml → release:canary-label), which refuses unpublished versions; live exercise tracked in verify(canary): exercise the canary label + note surface on two real 0.0.4 canaries #1149
• Landed and ratified before any 0.0.5 delivery work begins (D2) *(owner ruling — ruling recorded 2026-08-03 on epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120 (comment + box annotation): D2 reads as orchestrated delivery, so the pre-ratification CI-scoping merges fix(ci): desktop-native-linux honors the classifier it depends on #1153/ci: scope every expensive job to a classifier capability vector (#1152) #1155 are outside scope; no milestone-run wave has been planned or dispatched; merging this ratified draft is the ratification act (D1)
• No publish mechanics are restated from netscript-release — ownership pointers only; verified by Sol cycles 1–4 after the C1 fix removed the restated doctrine (grep hits only the ownership disclaimers)
• Every gate introduced is demonstrated to be capable of firing, with the negative case shown — per-row firing citations in the pre-merge gate table; the two initially narrower rows are demonstrated in the run's gate-demos.md (check 3: synthetic RED exit 1 / GREEN exit 0; the ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142 selection rule recovering PR ci: scope every expensive job to a classifier capability vector (#1152) #1155's true pre-merge SUCCESS from a live rollup), both independently re-verified by Sol cycle 3; the drift gate's negative case is on record from feat(release): canary label surface — derive the label from the published version, and make the payload content-derived #1121; fix(release): canary drift check is not target-scoped, so every historical canary reports as missing a label #1160 is closed (lineage note)
• The 0.0.5 milestone runs on this system as its first real execution (observational — route — routed: verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 (milestone 0.0.6) owns the proof with four checkable criteria

Validation

• deno task agentic:sync-claude — SYNCED: 18 skills, 22 mirrored files
• deno task agentic:check-claude — all checks OK (re-run green after d1a3e21c0)
• Sol review_claude adversarial pass — cycle 1: CHANGES_REQUESTED (9C/6M → fixed,
  d1a3e21c0); cycle 2: CHANGES_REQUESTED (5 fixed/demonstrated in bf1f8456f, 2 escalated;
  owner ruled 2026-08-03, recorded on epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120); cycle 3: CHANGES_REQUESTED on two residues
  (fixed, eb833401d) with all substance resolved and both gate demos independently re-verified;
  cycle 4: PASS, zero findings. Merging under the owner's standing merge-on-green directive.
• Docs-only diff → ci:skip-e2e + ci:skip-scaffold applied intentionally

Harness

• Run dir: .llm/runs/feat-milestone-orchestrator-artifacts--authoring/
• Phase: impl-eval — Sol thread 019fc874-c9b4-7b43-8af7-8abc6d6dae8d; see phase comments.

Drift / Debt

• Lane + eval substitutions and the cycle-1 drift record live in the run's drift.md:
  documentation-authoring exception (CLAUDE.md, 2026-06-18); owner directive routing ratification
  through the Sol adversarial pass; D2 tension surfaced (fix(ci): desktop-native-linux honors the classifier it depends on #1153/ci: scope every expensive job to a classifier capability vector (#1152) #1155).

🤖 Generated with Claude Code

https://claude.ai/code/session_01ReZGc3KP8xvEuruz1io7Pq

[rickylabs]

[PHASE: PLAN]

Outline locked before body-writing, per the brief. Full plan of record:
.llm/runs/feat-milestone-orchestrator-artifacts--authoring/plan.md.

The line between the artifacts

• Skill — the role: decisions (cluster, sequence, delegate, merge authority, when a canary
  goes out). Never contains run artifacts, gate lists, or the label mechanism.
• Profile (workflow/milestone-run.md) — the run: proof (layout, stage contracts, the
  pre-merge gate with per-gate firing evidence, cut-time checklist, evaluator protocol, DoD).
  Never contains role judgement, routing tables, or the label mechanism.
• Cadence (workflow/canary-cadence.md) — the schedule: trigger, membership, D3 identity,
  note, drift gate — wired to release:canary-label (feat(release): canary label surface — derive the label from the published version, and make the payload content-derived #1121/feat(release): derive canary labels from published versions #1122), never reimplemented.

Shared stories appear once (e.g. the #1086 mid-wave falsification lives in the cadence doc; the
others reference it). Every rule carries [observed] (0.0.4 cut-trace) or [asserted].

Section headings

Skill: When to Use / When Not to Use · Evidence discipline · Reading a milestone into PR
clusters · Wave sequencing and dispatch · Re-planning is normal · Delegation and effort tiering
(defers to lane-policy) · Merge authority · When a canary goes out (defers to canary-cadence) ·
Honesty rules · Supervision pitfalls · Reference files

Profile: When to use · Run layout (incl. instrumented cut-trace.md) · Stage contracts
(bootstrap → wave plan → dispatch → wave landing → canary point → loop → cut → close) · The
pre-merge gate [observed] · Gate integrity rules (proof-of-firing, pass ≠ did-not-run, the #1142
false-red trap, serialised expensive gates) · Cut-time checklist [observed] · Evaluator protocol ·
Definition of done · What varies per run · Checklist

Cadence: What this owns (+ #1119 disambiguation) · Evidence base · Trigger: the wave boundary
[observed] · Membership: content-derived [observed] · Identity: the label is the published version
(D3) · The canary note · The drift gate (negative case; #1160 known limitation) · Flexibility
[observed] · Open questions [asserted] · Reference

CI lane: docs-only diff → ci:skip-e2e + ci:skip-scaffold applied intentionally.

[rickylabs]

[PHASE: IMPL] S1 landed — workflow/canary-cadence.md (1774f6c). Trigger [observed], membership [observed] (the #1086 falsification is canonically homed here), D3 identity + note contract wired to release:canary-label, drift gate with firing evidence (#1121) and did-not-run signature (five pre-allocated not run check records), #1160 cited as a known limitation, #1119 disambiguated. Open questions kept [asserted]/owner-undecided.

[rickylabs]

[PHASE: IMPL] S2 landed — workflow/milestone-run.md (04efa4b). Stage contracts A–G with cut-trace.md as the signature artifact; the 7-item pre-merge gate as a table of check → firing evidence → did-not-run signature; gate integrity rules (proof-of-firing, #1142 latest-run-per-check, serialised expensive gates, honesty rule with #1092/#1146 precedents); cut-time checklist; evaluator protocol incl. the scoped reviewer-substitution waiver. PLAN-EVAL of the wave plan explicitly marked [asserted] — 0.0.4 ran without one.

[rickylabs]

[PHASE: IMPL] S3 landed — agent-milestone-orchestrator skill + generated mirror (5268abe). Role judgement only; gate lists / label mechanism / routing referenced, never restated. Validation: agentic:sync-claude SYNCED (18 skills, 22 files — ridealong regeneration of aspire + netscript-release mirrors that were stale on main, +22/−6); agentic:check-claude all checks OK. fmt note: repo fmt scope is packages/plugins TS only, so the authored Markdown is outside fmt jurisdiction; house 100-col style applied.

[rickylabs]

Acceptance evidence (v1 — superseded by the PR body after Sol eval cycle 1)

The corrected mapping lives in the PR body §Acceptance evidence. Kept for the record: the original [PHASE: IMPL] summary said all four slices landed, which remains true; the evidence mapping it carried was revised per Sol findings C8/C9.

[rickylabs]

[PHASE: REVIEW] [VERDICT: CHANGES_REQUESTED]

VERDICT: CHANGES_REQUESTED

Findings

• C1 .llm/harness/workflow/canary-cadence.md:150 — The cadence restates publish doctrine that
  netscript-release owns: a failed canary is evidence rather than an incident and is not yanked by
  default; lines 12–15 additionally enumerate the green-pair, same-semver-republish, and rollback
  mechanics — epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120 permits an ownership pointer, not copied mechanics, and
  .agents/skills/netscript-release/SKILL.md:89-134 is already the canonical statement.
• C2 .llm/harness/workflow/canary-cadence.md:118 — The cadence restates
  canary-label.ts internals: all five private check-record names, up-front not run allocation,
  the CLI invocation, bidirectional drift algorithm, and release mutation behavior — those details
  are implemented at .llm/tools/release/canary-label.ts:214-249,360-377,416-441; the run's own
  context-pack.md:12-14 explicitly forbids label-mechanism internals in these artifacts.
• C3 .llm/harness/workflow/canary-cadence.md:101 — “Canary notes accumulate into the stable
  release note” is a settled but nonexistent mechanism, repeated by milestone-run.md:108-111 —
  .llm/tools/release/github-release.ts:529-537 composes the stable body from a manual intro,
  GitHub-generated changes, and independently fetched closed issues; it never reads or accumulates
  canary releases, and neither artifact defines the missing verification.
• C4 .llm/harness/workflow/canary-cadence.md:109 — The claim that
  release:canary-label “ends every run with a drift check” is false — the implemented --dry-run
  path returns at .llm/tools/release/canary-label.ts:509-512 before label application, note
  publication, or drift, leaving those pre-allocated checks NOT_RUN.
• C5 .llm/harness/workflow/canary-cadence.md:67 — The documented standalone command does not make
  a hand-typed/wrong-current version “impossible by construction” — parseArgs requires the caller
  to supply --published-version (canary-label.ts:416-441) and assertPublishedCanary only proves
  that the supplied value exists somewhere in registry history (:187-195); the machine-derived
  identity exists in .github/workflows/release-canary.yml:121-131, but the cadence replaces that
  integration with a manual placeholder invocation.
• C6 .llm/harness/workflow/milestone-run.md:51 — Provider quota and paid-transport verification
  are explicitly promoted to dispatch gates without a pass record, a did-not-run signature, or a
  mechanism that can distinguish either state — cut-trace.md:61-68 supplies the historical
  negative costs only, and the later per-merge table does not cover these two gates.
• C7 .llm/harness/workflow/milestone-run.md:64 — The assertion that every pre-merge item has fired
  is false for check 3 — its cited “negative case” at lines 74–75 and cut-trace.md:76-78 proves only
  that quoted strings in excluded paths cause false positives; it does not demonstrate the stated
  predicate firing on a new eligible deno-lint-ignore, as unknown as, or @ts-ignore.
• C8 .llm/runs/feat-milestone-orchestrator-artifacts--authoring/worklog.md:50 — The acceptance map
  checks “a fresh orchestrator can run a milestone” while simultaneously deferring the run-from-it
  proof to verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 — verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 says it “owns the proof that they work” and all four of its criteria remain
  unchecked, so the PR is closing epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120 on an unproven criterion in direct conflict with
  milestone-run.md:98-102's honesty rule.
• C9 .llm/runs/feat-milestone-orchestrator-artifacts--authoring/worklog.md:50 — The D2 acceptance
  mapping claims that ratification precedes all 0.0.5 delivery “by construction,” but GitHub shows
  unrelated 0.0.5-milestone PRs fix(ci): desktop-native-linux honors the classifier it depends on #1153 and ci: scope every expensive job to a classifier capability vector (#1152) #1155 merged at 15:30 and 15:49, before feat(harness): agent milestone orchestrator skill, milestone-run profile, rolling canary cadence #1161 was created
  at 16:13 and while it remains unratified; the owner amendment allowed only the thin canary
  label/publish/observability slice ahead of these artifacts, not general CI delivery.
• M1 .agents/skills/agent-milestone-orchestrator/SKILL.md:42 — Multiple rules are promoted as
  [observed] although the declared evidence source does not contain them: supervisor-per-PR
  identity (line 42), workflow blocking/host load 160 (58–60), attached-launch behavior (98–100),
  evaluator triggers (101–104), truncated-log incident (112–113), worktree ownership matching
  (146–148), and AppHost stop claims (149–150) are absent from cut-trace.md:1-103.
• M2 .llm/harness/workflow/milestone-run.md:120 — The [observed] evaluator section is not backed
  by the cut trace: draft→ready augment behavior, label-triggered OpenHands, docs(fresh): repair imports, forms, and render-ui discovery #1113's three-family
  path, owner-review substitution, and the inline-jq finding do not appear in
  cut-trace.md:1-103; external examples may justify an assertion, but cannot satisfy this
  artifact's definition of [observed] as trace-derived.
• M3 .llm/harness/workflow/canary-cadence.md:77 — The repo-version/publish-result trap is marked
  [observed], but the 0.0.4 cut trace contains no repo version, canary target, publish result, or
  label execution; the ratified design itself calls this “found while scoping,” so it has been
  promoted beyond the stated empirical source.
• M4 .llm/harness/workflow/milestone-run.md:92 — The “latest run per check name before merge”
  audit algorithm is presented as a settled gate-integrity rule without proof-of-firing — ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142
  establishes a post-merge false-red defect and proposes fixing/short-circuiting the classifier,
  but neither ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142 nor the cut trace establishes this new timestamp-selection algorithm.
• M5 .agents/skills/agent-milestone-orchestrator/SKILL.md:98 — A fresh orchestrator still cannot
  execute or supervise stage C from the three artifacts: they say only “launch attached” and retain
  a threadId, but omit the repo's required agentic launcher, the codex-watch --mode turn
  interception rule called out by epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120, the follow-up steering path, and any reference to
  workflow/tooling.md or workflow/agent-handoff.md; this is a load-bearing operational gap, not
  routing-table duplication.
• M6 .llm/runs/feat-milestone-orchestrator-artifacts--authoring/context-pack.md:20 — The resumable
  state is knowingly stale: it says S4 is next, while worklog.md:45-52, PR feat(harness): agent milestone orchestrator skill, milestone-run profile, rolling canary cadence #1161's acceptance
  comment, and head commit aa11f0b33 all record S4 complete; a resumed supervisor could duplicate
  the verification issue/comment external actions.

[rickylabs]

[PHASE: REVIEW-TRIAGE]

Cycle-1 verdict: CHANGES_REQUESTED (9C/6M). All 15 findings accepted, none declined. Fixes in
d1a3e21c0; per-finding dispositions in the run's worklog.md §Eval cycle 1. Highlights:

• C1/C2 cadence reduced to ownership pointers + observable contract (no netscript-release
  doctrine, no canary-label.ts internals).
• C3 note→stable-note accumulation corrected to what exists: canary notes are the cut's
  source material, read manually — marked [asserted], gap stated (cadence + cut checklist).
• C4/C5 dry-run semantics stated truthfully; identity derivation re-anchored on the
  release-canary.yml wiring (publish output → tool); standalone CLI bounds, does not derive.
• C6/C7/M4 gates given honest proof forms: quota/transport = recorded check output before
  dispatch; pre-merge check 3's predicate and the ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142 mitigation explicitly marked
  not-yet-demonstrated.
• C8 epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120 box 1 carries a relocation note (run-from-it proof → verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163, verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090/feat(release): canary label surface — derive the label from the published version, and make the payload content-derived #1121
  precedent); evidence re-scoped.
• C9 D2 tension surfaced in the PR-body evidence: fix(ci): desktop-native-linux honors the classifier it depends on #1153/ci: scope every expensive job to a classifier capability vector (#1152) #1155 (0.0.5 milestone) merged
  pre-ratification; the owner's merge ratifies the narrower reading or the criterion moves.
• M1/M2/M3 [observed] redefined as the recorded 0.0.4 execution with per-claim citations
  (trace / filed issues / ratified design record); repo-version trap re-attributed as a scoping
  finding.
• M5 stage-C operability closed: tooling.md + agent-handoff.md wired into skill and
  profile; codex-watch --mode turn interception rule added.
• M6 context-pack corrected with a do-not-repeat-S4 guard.

Re-review requested on the same Sol thread (eval loop 1 of 2).

[rickylabs]

[PHASE: REVIEW] [VERDICT: CHANGES_REQUESTED]

Findings

• C3 (survives) .llm/harness/workflow/canary-cadence.md:166 — The reference table still claims
  milestone-run.md performs “cut-time verification of note accumulation,” preserving the
  nonexistent mechanism C3 identified — the corrected contract at cadence lines 102–111 and
  milestone-run.md:113-118 says the notes are only read manually and no mechanism feeds them into
  the stable note.
• C7 (survives) .llm/harness/workflow/milestone-run.md:76 — Pre-merge check 3 remains an active
  gate although its negative predicate has never been observed firing — this directly violates the
  profile's own admission rule at lines 86–87 and epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120's acceptance requirement that every gate's
  negative case be shown. The PR body's checked acceptance row does not become true by disclosing
  the failure; it explicitly admits this predicate is not demonstrated.
• C9 (survives) PR #1161 body, Acceptance evidence D2 — D2 is still checked despite evidence in
  the same sentence that 0.0.5 delivery PRs fix(ci): desktop-native-linux honors the classifier it depends on #1153/ci: scope every expensive job to a classifier capability vector (#1152) #1155 merged before ratification — issue epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120
  says no 0.0.5 delivery work begins first, and its only owner amendment narrowly permits the
  canary surface. A future merge cannot retroactively ratify an unrecorded “orchestrated delivery”
  rewrite, so the conditional mapping is not truthful acceptance evidence.
• C10 .llm/harness/workflow/canary-cadence.md:94 — The revised observable contract falsely says
  the tool enforces publication on an existing v<version> tag —
  canary-label.ts:360-377 only looks up an existing release and, on 404, POSTs a release with that
  tag; it never verifies that the git tag already exists. This is new mechanism drift introduced by
  the fix wording.
• M1 (survives) .agents/skills/agent-milestone-orchestrator/SKILL.md:33-36 — The fix changes the
  meaning of [observed] to include issue prose and design observations instead of backing every
  marker with the cut trace as this review's source-of-record rule requires. The unsupported earned
  rules remain, including host load 160 (lines 60–62), attached-launch behavior (101–108), evaluator
  triggers (109–112), truncated-log behavior (120–122), and AppHost stop behavior (159–161), none of
  which appears in cut-trace.md.
• M2 (survives) .llm/harness/workflow/milestone-run.md:127-142 — The evaluator section remains
  promoted as [observed] without cut-trace support for draft→ready/OpenHands triggers, docs(fresh): repair imports, forms, and render-ui discovery #1113's
  three-family path, reviewer substitution, or the inline-jq incident. Broadening the document's
  evidence definition at lines 14–16 does not satisfy the required trace verification.
• M4 (survives) .llm/harness/workflow/milestone-run.md:94-100 — The latest-run-per-check timestamp
  algorithm remains a mandatory “must” rule while simultaneously marked [asserted] and
  not-yet-exercised — asserted content is still phrased as settled doctrine, contrary to the same
  document's proof-before-entry rule at lines 86–87.
• M7 .llm/harness/workflow/canary-cadence.md:129-131 — The cadence says target-scoped drift is
  “not yet” implemented and cites fix(release): canary drift check is not target-scoped, so every historical canary reports as missing a label #1160 as a live defect, but fix(release): canary drift check is not target-scoped, so every historical canary reports as missing a label #1160 is closed with all four
  acceptance boxes checked. The reference is stale and no longer says what the artifact claims.

Cycle-1 findings C1, C2, C4, C5, C6, C8, M3, M5, and M6 are resolved. Exact-paragraph comparison
found no duplicate paragraph across the three artifacts, and the generated orchestrator mirror is
byte-identical to its source.

[rickylabs]

[PHASE: REVIEW-TRIAGE] (cycle 2 — second CHANGES_REQUESTED → escalated per the two-failure eval rule)

Sol confirms C1/C2/C4/C5/C6/C8/M3/M5/M6 resolved, zero duplicate paragraphs, mirror
byte-identical. Of the 8 cycle-2 findings, 5 are fixed in bf1f8456f:

• C3 residue — reference-table row corrected ("reads the canary notes manually").
• C10 — tag-existence wording corrected against the tool's actual behaviour.
• M7 — fix(release): canary drift check is not target-scoped, so every historical canary reports as missing a label #1160 is closed; the stale "known limitation" is now a lineage note.
• C7 — resolved by demonstration: gate-demos.md § Demo 1 — check 3's predicate goes RED
  (exit 1) on a new ignore in publishable source, GREEN on excluded-path quotes, GREEN on a clean
  diff. The negative case is now shown, not promised.
• M4 — resolved by demonstration on live data: gate-demos.md § Demo 2 — merged PR ci: scope every expensive job to a classifier capability vector (#1152) #1155's
  rollup carries a post-merge classify changes FAILURE (+28s) and a superseded CANCELLED; both
  clauses of the rule fire and recover the true pre-merge SUCCESS.

Escalated to the owner (merge stays blocked):

1. C9 / D2 — fix(ci): desktop-native-linux honors the classifier it depends on #1153/ci: scope every expensive job to a classifier capability vector (#1152) #1155 (milestone 0.0.5) merged before this PR existed. The D2 evidence box
   is now unticked. Owner picks: (a) record on epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence #1120 that D2 means orchestrated delivery —
   box becomes tickable; or (b) D2 is violated-as-written — criterion moves/annotates per the
   honesty rule.
2. M1/M2 / [observed] source of record — Sol holds the marker must be cut-trace-only. The
   artifacts follow the ratified design doc's own practice: trace + issues filed by the run
   (docs(fresh): repair imports, forms, and render-ui discovery #1113, agentic: no way to see what a running agent is doing — add follow + live state for Codex and agy #1115) + dated design observations, each cited at the claim. Demoting filed-issue
   observations to "asserted/unproven" would be less accurate, not more honest. Owner ratifies
   the wider definition or orders the demotion.

Per the owner's directive the verdict gate stays: no green, no merge.

[rickylabs]

[PHASE: REVIEW] [VERDICT: CHANGES_REQUESTED]

Findings

• C10 (survives) .llm/harness/workflow/canary-cadence.md:94-97 — The edit now admits that
  release:canary-label does not verify prior tag existence, but immediately claims its
  published-version guard “in practice means the tag exists before a note can.” That implication
  is still false: assertPublishedCanary proves only that the JSR version exists, while
  publishCanaryRelease (.llm/tools/release/canary-label.ts:360-377) neither reads nor verifies a
  git tag before POSTing the GitHub release. Registry publication is not tag-existence evidence.
• M8 PR #1161 body, Acceptance evidence — Every gate — The re-ticked acceptance mapping was not
  updated to the evidence it claims: it still says check 3 and the ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142 mitigation are
  “not-yet-demonstrated” and still calls fix(release): canary drift check is not target-scoped, so every historical canary reports as missing a label #1160 a “known limitation.” At head, check 3 and ci(e2e-cli): classify changes corrupts $GITHUB_OUTPUT post-merge — every merged PR shows a false red #1142 are
  claimed demonstrated by gate-demos.md, and canary-cadence.md:131-134 says fix(release): canary drift check is not target-scoped, so every historical canary reports as missing a label #1160 is fixed.
  The checked mapping is therefore internally contradictory and stale even though the underlying
  demonstrations pass review.

Cycle-2 findings C3, C7, C9, M1, M2, M4, and M7 are resolved. C7's predicate independently
returned RED/exit 1 for a new ignore in publishable source and GREEN/exit 0 for an excluded-only
hunk. M4's recorded #1155 rollup exactly matches GitHub: latest pre-merge classify changes was
SUCCESS at 15:42:36Z, followed after the 15:49:19Z merge by CANCELLED and FAILURE runs. C9 and
M1/M2 resolve under the owner rulings recorded on #1120. Exact-paragraph comparison remains clean,
and all three regenerated skill mirrors are byte-identical to their sources.

[rickylabs]

[PHASE: REVIEW] [VERDICT: PASS]

Findings

• None.

C10 is resolved: .llm/harness/workflow/canary-cadence.md:94-99 now states that the canary cut
creates the tag, that release:canary-label does not verify it, and that an absent tag would make
GitHub create it at default-branch HEAD. This matches .llm/tools/release/canary.ts:188-190 and
.llm/tools/release/canary-label.ts:360-377; the published-version refusal is accurately limited
to version identity rather than tag existence. M8 is resolved: PR #1161's current “Every gate”
acceptance row cites both demonstrations in gate-demos.md, records their independent cycle-3
verification, and identifies #1160 as closed/fixed lineage. The three-file eb833401d diff
introduces no new defect in the scoped surfaces.

[rickylabs]

[PHASE: IMPL-EVAL] [VERDICT: PASS]

Eval of record closed green: Sol review_claude (xhigh) cycle 4 = PASS, zero findings, after cycles 1–3 (all 17 distinct findings fixed or demonstrated; two owner rulings recorded on #1120). Evidence mirror dry-run validates all 8 acceptance mappings (acceptance-mirror DRY-RUN: #1120 (8)). Marked ready and moved status:impl-eval → status:ready-merge in this action; merging on green close-gate per the owner's standing directive.

[rickylabs]

@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=400

use harness

SKILL

• netscript-harness — preserve evaluator separation and record a concise, evidence-based verdict.
• netscript-tools — use repository-native commands and keep lock/worktree state clean.
• openhands-handoff — honor the pr-comment output contract and write the required summary.
• netscript-cli — hand-test scaffold and documented CLI commands when executable claims exist.
• rtk — keep changed-file and command inspection compact.

Act as a cheap-and-quick documentation accuracy evaluator. Do not edit source, documentation,
deno.lock, or run artifacts, and do not create commits. Write the complete result to
OPENHANDS_SUMMARY_PATH; output mode pr-comment will publish that result as the single PR
comment.

1. Identify every changed documentation file in this PR and read each one fully. Include Markdown,
   MDX, READMEs, RFCs, harness/process docs, and skill instructions; do not review only the diff
   hunks.
2. For every executable claim, QUICKLY hand-test the exact documented command or snippet from the
   repository root. Use the smallest representative scaffold/workspace needed, compare real output
   and generated paths with the document's claim, and stop after the highest-value small checks. Do
   not substitute a full expensive E2E suite for a focused manual check.
3. If the changed set contains no executable command, snippet, flag, verb, or path claim, say
   exactly:
   No executable documentation claims in this changed set; manual command testing was not applicable.
   Full accuracy and hallucination review is still mandatory.
4. Check every named command verb, flag, file path, package, API, and expected output against the
   repository. Any hallucinated or nonexistent verb, flag, or path is a BLOCKING finding.
5. Return a compact per-file table with accurate, inaccurate, or unverifiable, the commands or
   snippets tested, observed-versus-claimed output, and blocking findings. End with one overall
   PASS or FAIL_FIX verdict and a short list of exact fixes.

Keep the iteration budget small. Prefer one to three decisive manual checks over broad exploration,
but never omit a changed documentation file from the per-file verdict table.

[rickylabs]

OpenHands Agent — Completed

OPENHANDS_VERDICT: PASS

Model: openrouter/minimax/minimax-m3
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=success replies=skipped draft-pr=skipped
Commit-back skipped 60 non-allow-listed path(s) (never pushed; see uncommitted.txt/uncommitted.patch in the Actions artifact): .llm/assets/agent-docs/provenance.json,deno.json,deno.lock,packages/ai/deno.json,packages/aspire/deno.json
Trace: .llm/tmp/run/openhands/pr-1161/run-30838335823-1 (in the Actions artifact)
Artifact: openhands-agent-30838335823-1

OPENHANDS_VERDICT: PASS

Documentation accuracy — feat(harness): milestone orchestrator + canary cadence

Per-file verdict table for the five changed documentation files in PR #1161.

File	Verdict	Commands / snippets tested	Observed vs claimed	Blocking findings
.agents/skills/agent-milestone-orchestrator/SKILL.md (175 lines)	accurate	deno task agentic:codex-watch source grep for --mode turn; deno task registry for every deno task agentic:*/release:* task referenced	--mode turn is a real CLI flag (line 75 of .llm/tools/agentic/codex/codex-watch.ts); its docstring on lines 4–12 says "wake when the agent's current turn FINISHES. Watches the thread's … task_complete" — matches the SKILL.md wording exactly. Every deno task the SKILL.md names (agentic:dispatch-openhands, agentic:codex-status, agentic:codex-watch, agentic:gh-pr, agentic:review-threads, agentic:leak-check, agentic:teardown, release:canary-label) is present in deno task output. .llm/tools/harness/watch-run.ts exists. All 5 GitHub issues it cites (#574, #1120, #1121, #1122, #1161) resolve with matching titles.	none
.llm/harness/workflow/milestone-run.md (174 lines)	accurate	deno task release:canary-label -- --help; deno task release:canary-label -- --published-version 0.0.5-canary.1 --head 26bd9a10... --repo rickylabs/netscript --dry-run	--help usage line is Usage: release:canary-label --published-version <x.y.z-canary.n> [--head <ref>] [--repo owner/name] [--json] [--dry-run] — matches every flag in milestone-run.md. Dry-run prints the exact 5-stage chain the doc cites (published-version → merge-history-payload → label-application → release-note-publication → drift) and fails the first stage with FAIL: @netscript/cli@0.0.5-canary.1 is not published followed by NOT_RUN: not run for the downstream stages — matches the doc's "stop after the published-version gate" claim. Every deno task cited (release:canary, release:canary-label, release:publish, release:verify-canary-pair, release:cut, release:preflight) is present.	none
.llm/harness/workflow/canary-cadence.md (174 lines)	accurate	grep against .llm/runs/release-0.0.4--orchestration/cut-trace.md for the falsification cases the doc paraphrases	The cut-trace contains the exact sentences canary-cadence.md quotes: "merge 5 (#1086) was not part of wave 2. It was dispatched mid-wave because #1089 blocked the owner's docs-audit lane, and it merged between two wave-2 PRs." Wave-cluster timings (21 min / 42 min / 2h01m), header "11 PRs, 42 issues closed, ~6h40m", and the four counter-observations (#1089, #1013, #1024/#1061, mid-wave splits) all match verbatim. The 0.0.4 trace's actual orchestration-begin → last-merge span is ~6h25m (08:34 → 14:59 local), well within the doc's tilde'd "~6h40m" — not a blocker. Every GH issue cited (#1004, #1011–#1024, #1039, #1048, #1056, #1061, #1064–#1071, #1074–#1078, #1080–#1094, #1100, #1101, #1120–#1122, #1158–#1161) resolves with matching titles. .github/workflows/release-canary.yml step "Label published canary payload and verify drift" wires --published-version to deno task release:canary-label (lines 118–128), matching the wiring the doc describes.	none
.claude/skills/aspire/SKILL.md (modified)	accurate	aspire stop --apphost <path> (path discovered at /home/runner/.aspire/bin/aspire, version 13.4.6); aspire ps --format Json	aspire stop --apphost <path> is a real CLI flag in aspire 13.4.6. aspire ps runs and returns valid JSON ([] when no AppHosts are running). The SKILL.md's new guidance "take the exact appHostPath from aspire ps and pass it to the stop command" matches aspire's actual JSON shape. The "host-wide aspire stop --all is dangerous / unreliable" claim is sourced from the diff context (it has been observed to report "No running AppHost found" while sibling processes survived).	none
.claude/skills/netscript-release/SKILL.md (modified)	accurate	direct read of lines 110–130 after the diff; cross-check against .llm/tools/release/canary-label.ts (19205 bytes, present)	Doctrine change is internally consistent: the old "yank the canary" default is replaced with "Do not yank the failed canary … A canary is a prerelease: normal semver resolution never selects it, so a failed one is inert for consumers and remains useful as audit evidence." The "yanking stays available for actively-harmful canaries" carve-out is preserved. The owner decision date (2026-08-02) predates the 0.0.4 cut trace (2026-08-03) by one day — internally consistent. The next-canary N selection rule ("Failed canary versions still count when choosing the next N") is consistent with canary-label.ts's monotonic version counting.	none

Summary

All five changed documentation files are accurate against the repository. Every executable claim (deno task verb, CLI flag, file path, GitHub issue number, GH workflow step) resolves to a real artifact. The one quantitative looseness — "~6h40m" vs the actual 6h25m orchestration span — is bounded by an explicit tilde and is not a blocker.

Overall verdict: PASS.

No fixes required.

Run: https://github.com/rickylabs/netscript/actions/runs/30838335823