epic: agent milestone orchestrator + rolling canary release system — skill, harness profile, cadence

[rickylabs]

Decisions — RATIFIED by the owner, 2026-08-03

• D1 — Instrument: draft PR, not an RFC. RFCs govern product surface; this is internal
  doctrine (skill, harness profile, orchestration cadence) and ships as a draft PR for
  ratification.
• D2 — 0.0.4 keeps the current release system. The cut runs on today's
  netscript-release mechanics. The agent milestone orchestrator ships immediately after
  the 0.0.4 cut, and 0.0.5 then starts on the new system as its first real execution.
• D3 — Canary labels follow semver prerelease form exactly: canary:<version>-canary.<n>
  — e.g. canary:0.0.5-canary.1. The label names the published prerelease version itself,
  so the GitHub view and the JSR version string are the same string. No separate ordinal
  vocabulary to reconcile.

Sequencing consequence

This work sits between the 0.0.4 cut and the start of 0.0.5 delivery. It carries the 0.0.5
milestone because 0.0.5 is what runs on it, but it is 0.0.5's first landed item and gates the
rest of the milestone — no 0.0.5 delivery work begins until the skill, profile and cadence
exist and the draft PR is ratified.

---

The gap

We publish canaries only to smoke-test an actual JSR release. Every mature framework ships
regular canaries after each grouped merge / sub-milestone / epic. We have the publish
mechanics and none of the cadence.

Concretely, there is today no answer to: when does a canary go out, what does it contain, what
merge order produced it, and where can anyone see that in GitHub?

The 0.0.3 → 0.0.4 orchestration run was the first real execution of the milestone-orchestrator
pattern, and it was carried entirely in one agent's head. Nothing about it is written down, so
the next orchestrator rediscovers it from scratch.

What to build — three artifacts

1. Skill — agent-milestone-orchestrator

The role. Reading a milestone into PR-sized clusters of linked issues, sequencing them into
small workflow waves, one supervisor per PR, delegation and effort tiering, merge authority, and
when a canary goes out.

2. Harness profile — the milestone/release run shape

The run. Run artifacts, gates, evaluator protocol, and what "done" means for a milestone run.
Skill is the role; profile is the run. If the same paragraph appears in both, it belongs in one
and is referenced from the other.

3. Rolling canary cadence

Canaries at pre-defined points that shape merge order and the workflow plan, not as an
afterthought. Requirements:

• Flexible. Priorities shift mid-flight. In the 0.0.4 run, fix(agentic): Antigravity adapter argv drops the prompt — --print swallows --print-timeout as its value #1089 jumped the queue because it
  blocked another lane, and a Gemini quota exhaustion stalled docs delivery entirely. The cadence
  must absorb that rather than break.
• Observable in GitHub. Anyone should see which canary an issue or PR is destined for, and
  which have shipped — see D3.
• Composed of what already works. Wrap the proven agentic system; add Claude workflow waves,
  Codex supervisors, agy research agents, and OpenCode — each restrained to where it genuinely
  earns its place, not used because it exists.

Non-duplication boundary — this is the main failure mode

Surface	Already owns	Do not restate
.agents/skills/netscript-release	How to publish a canary safely — publish:readiness, release:preflight, OIDC, the green canary pair, the import-attribute ban and its incident lineage, rollback	All publish mechanics
.agents/skills/netscript-harness	The general harness operating model	Run mechanics in general
.llm/harness/workflow/lane-policy.md	Routing: provider/model/effort per lane	Any second routing table
.llm/harness/archetypes/*	Per-archetype design profiles	Archetype content
.llm/tools/agentic/*	Launching, watching, PR lifecycle, leak-check/teardown	Tool internals

Learnings to encode — verify each, do not take on faith

Orchestration:

• Launch Codex app-server-attached, never ad-hoc codex exec. An attached thread takes
  further turns via turn/start with its threadId; a plain exec is one-shot and unreachable.
  An hour was lost to this on the docs agent.
• Liveness is not progress, and artifacts are not always where you launched. A research agent
  believed to have produced nothing in 70 minutes was 25/27 complete, writing into per-sub-agent
  worktrees. It came within one command of being killed. Filed as agentic: no way to see what a running agent is doing — add follow + live state for Codex and agy #1115.
• codex-watch --mode turn is the clean interception point — it fires on task_complete,
  when the agent is idle between turns. Git activity is not that signal.
• Small workflow waves. A workflow blocks until every agent in it completes, so a large
  fan-out stops the next block starting — and it is what freezes WSL. Load hit 160 at peak.
• One supervisor per PR, each PR closing a group of linked issues. Avoids micro-PRs, and
  avoids an agent blocked because closing its issue depends on another.
• Evaluate only when necessary. Draft→ready already triggers augment; a label auto-triggers
  OpenHands. Spawning a local evaluator per PR is waste.
• Generator ≠ evaluator, and the supervisor is not the evaluator either. docs(fresh): repair imports, forms, and render-ui discovery #1113 was written by
  Gemini, supervised by Codex, evaluated by a third family before merge.
• Quota exhaustion is a first-class failure mode. The docs lane hit the Gemini cap mid-delivery
  and correctly refused to silently substitute another model. A cadence that assumes model
  availability will break. Provider quota belongs in the plan, not the postmortem.

Verification and hygiene:

• Verify the artefact, never the exit code. Three separate agents claimed to have stopped
  their AppHost; all three had left the process tree running.
• aspire stop --all reports "No running AppHost found" and exits 0 while its tree lives on,
  and agentic:teardown reports stoppedAppHosts: [] for the same tree.
• Predicate bugs are the signature defect of this kind of work. The 0.0.4 run shipped two
  guards whose condition could never fire: a watcher requiring non-draft when every PR was a
  draft, and an origin/main..HEAD ancestry check that is wrong under squash-merge because merged
  commits are never ancestors. Both did nothing and looked correct. Every gate this system adds
  must be proven to be able to fire.
• Absence of red is not green. For each gate, state what it reports when it does not execute
  at all — and ensure pass is distinguishable from did-not-run.
• Observational acceptance criteria cannot be closed by a PR. Criteria of the form "a
  follow-up agent run shows…" need a separate verification issue in the next milestone — the
  pattern invented during this run as verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090.
• Milestone bookkeeping: Closes #N belongs in the PR body; milestone open_issues
  counts PRs as well as issues; renaming milestones must run in descending order to avoid title
  collisions.

Acceptance

• agent-milestone-orchestrator skill exists and a fresh orchestrator can run a milestone
  from it without rediscovering the pattern (the run-from-it half is observational — proof
  relocated to verify(milestone-run): 0.0.5 runs on the orchestrator skill + milestone-run profile as its first real execution #1163 per the verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090 pattern; this box's PR-satisfiable scope is that the skill
  exists and encodes the pattern with cited evidence)
• A milestone/release harness profile exists, distinct from the skill, with no duplicated
  paragraph between them
• Canary cadence is documented, with defined trigger points tied to merge order
• Canary destination is visible in GitHub on issues and PRs using canary:<version>-canary.<n>
  (D3), and the label string matches the prerelease version actually published to JSR
• Landed and ratified before any 0.0.5 delivery work begins (D2) (owner ruling
  2026-08-03: "delivery work" reads as orchestrated delivery — work planned or dispatched
  through the milestone-run system; the pre-ratification CI-scoping merges fix(ci): desktop-native-linux honors the classifier it depends on #1153/ci: scope every expensive job to a classifier capability vector (#1152) #1155 are
  outside D2's scope)
• No publish mechanics are restated from netscript-release
• Every gate introduced is demonstrated to be capable of firing, with the negative case shown
• The 0.0.5 milestone runs on this system as its first real execution (observational — route
  to a verification issue in 0.0.6, per the verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090 pattern)

Provenance

Owner request, 2026-08-03: "we're barely leveraging canary release… we definitely need a proper
new skill anyway for 'agent milestone orchestrator' + a complete new harness profile (beware to
make sure each fulfils its purpose without duplication)."

Plan produced by the 0.0.4 orchestrator; this issue is the GitHub-visible record of it, filed
because the plan previously existed only in a local brief and an agent session.

Related: #1090 (observational verification pattern) · #1115 (no live agent visibility)

[rickylabs]

Owner ratified all three, 2026-08-03. Issue body updated to match.

• D1 — draft PR. Not an RFC. Internal doctrine, so it does not join the RFC: Frontend Contribution Layer — plugins that ship UI (dashboard · auth · ai · deploy) #890/RFC: NetScript Deploy Plugin Family — plugin-deploy core + per-cloud adapters, Deno-native first (#824) #891/RFC: NetScript Single Deployment — process-managed apps, single & multi runtime (#820) #822 queue.
• D2 — 0.0.4 keeps the current release system. The cut runs on today's netscript-release
  mechanics. The orchestrator ships right after the 0.0.4 cut, and 0.0.5 starts on the new
  system.
• D3 — canary:<version>-canary.<n>, exact semver prerelease form — e.g.
  canary:0.0.5-canary.1.

D3 is better than what I proposed. My canary:0.0.5-c1 invented a second vocabulary that would
have needed translating to the published version string every time. The owner's form makes the
label the version — the GitHub label and the JSR prerelease are literally the same string, so
there is nothing to keep in sync and no way for them to drift apart. Whoever builds the cadence
should treat that identity as a constraint on the label generator, not a convention: the label
should be derived from the version that was (or will be) published, never typed by hand.

D2 also sharpens the scope. This is 0.0.5's first landed item and it gates the rest of the
milestone — it keeps the 0.0.5 milestone because 0.0.5 is what runs on it, but no 0.0.5
delivery work starts until it is landed and the draft PR is ratified. The 0.0.4 cut therefore
becomes the observed case the cadence is designed against, not the first case it is tested on.

[rickylabs]

D2 amended by the owner, 2026-08-03 — 0.0.4 now exercises the canary surface

What changed. D2 originally made 0.0.4 the observed case only: the cadence would be designed
by watching the cut, and applied for the first time at 0.0.5. The owner has amended this to use
0.0.4 to exercise the canary surface for real, in the window while the cut is gated on docs
delivery.

Scope of the amendment is deliberately narrow. The canary system itself — label generation,
the label/version identity, cadence trigger points, GitHub observability. Not the full agentic
milestone-orchestration system. The 0.0.4 cut still runs its publish mechanics on today's
netscript-release, exactly as ratified in D2. What changes is that the thin canary surface gets
real use before 0.0.5 depends on it.

Sequence:

1. The canary-surface slice lands.
2. Canary 1 of 0.0.4 is cut using it — real publish, real label (canary:0.0.4-canary.1).
3. Docs agent lands its delivery.
4. Canary 2 of 0.0.4 — different payload, incremented label (canary:0.0.4-canary.2).
5. Stable 0.0.4, which remains the owner's explicit call.

Consequences for the design

• Separability is now a design constraint. The canary label/publish/observability surface must
  be a self-contained slice that can land and be used ahead of the skill and the harness profile.
  If it cannot be exercised without the orchestrator skill existing, the split is wrong.
• The first slice must stay thin. Landing first makes it the natural magnet for everything
  else; anything not needed to label, publish and observe a canary belongs in a later slice.
• Two live canaries replace a reasoned argument as proof-of-firing. The acceptance criteria
  above require every gate to be demonstrated capable of firing with the negative case shown —
  written because 0.0.4 shipped two guards whose condition could never fire. There are now two real
  executions with genuinely different payloads to prove that against. What each gate reported, and
  whether pass is distinguishable from did-not-run, should be recorded from those runs.
• First real test of the D3 identity constraint. If the label string and the prerelease version
  actually published to JSR ever diverge during this exercise, that is a finding — not something to
  reconcile by hand.