feat(#4718): add pre-script output protocol to fullsend run - #5737
Conversation
PR Summary by QodoAdd pre-script skip protocol to
AI Description
Diagram
High-Level Assessment
Files changed (6)
|
|
🤖 Review · ❌ Terminated · Started 5:58 PM UTC · Ended 6:00 PM UTC |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
🤖 Finished Review · ❌ Failure · Started 5:58 PM UTC · Completed 6:00 PM UTC |
Code Review by Qodo
1.
|
2aa519f to
9584a08
Compare
|
🤖 Review · |
Site previewPreview: https://0b1908a3-site.fullsend-ai.workers.dev Commit: |
9584a08 to
a012fbe
Compare
|
🤖 Review · |
a012fbe to
eded4af
Compare
|
🤖 Review · |
eded4af to
9d49198
Compare
|
🤖 Finished Review · ✅ Success · Started 7:08 PM UTC · Completed 7:22 PM UTC |
ReviewFindingsLow
Previous runReviewFindingsLow
Previous run (2)ReviewFindingsLow
Labels: PR adds pre-script skip protocol to the harness run lifecycle |
|
🤖 Review · |
0d3332a to
b5c2726
Compare
|
🤖 Finished Review · ✅ Success · Started 7:48 PM UTC · Completed 8:05 PM UTC |
Pre-scripts can now request that the run stop before sandbox creation by writing skipped=true (plus an optional reason=...) to the file named by the new FULLSEND_PRESCRIPT_OUTPUT env var. fullsend run parses the file after the script exits, reports a skipped completion status carrying the reason, relays the outputs to GITHUB_OUTPUT when running under GitHub Actions, and exits 0. This is the CLI half of the direction agreed on #4718: with skip gating living in the CLI, the reusable workflows' inline pre-script calls (the source of the double execution, and the last consumer of the scaffold script copies — #5667) become removable, and every forge inherits the gating without CI-side reimplementation. Version skew is handled asymmetrically, deliberately: a script that never writes the file proceeds exactly as before (fails safe), while a protocol-aware script under an older CLI finds the env var unset and cannot signal a skip, so the agent runs anyway (fails open). Scripts must guard on the variable, the same pattern as the existing GITHUB_OUTPUT guard. Malformed output is a hard error so a mistyped skip cannot silently proceed as a duplicate agent run — including values that differ from a reserved key only by case, and values carrying control characters that could smuggle extra GITHUB_OUTPUT entries. The normative contract lives in docs/normative/prescript-output/v1. Assisted-by: Claude (fix, review), Grok (review) Signed-off-by: Wayne Sun <gsun@redhat.com>
The PR mixed three names for the same thing — skip protocol, FULLSEND_PRESCRIPT_OUTPUT protocol, and contract. Standardize on "pre-script output protocol" (matching docs/normative/prescript-output/) as the proper name, with "contract" kept only as a generic descriptor in the normative doc's opening line. No behavior change. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
b5c2726 to
d08da49
Compare
|
🤖 Finished Review · ✅ Success · Started 8:47 PM UTC · Completed 9:03 PM UTC |
|
🤖 Finished Retro · ✅ Success · Started 9:31 PM UTC · Completed 9:38 PM UTC |
Retro: PR #5737 — Pre-script skip protocolWorkflow went reasonably well. A human-authored PR (waynesun09) adding a pre-script output protocol to Timeline
Key metrics
Existing issues with new evidence
What went well
|
The skip-flag convention this ADR originally recorded was withdrawn after review (#5013 and agents#175 closed as superseded). Rewrite the ADR — still unmerged, so immutability does not apply — to record the decision that replaced it: pre-scripts run exactly once, inside fullsend run, signalling skips via the pre-script output protocol shipped in #5737, with the field-level contract in docs/normative/prescript-output/v1. Addresses the review feedback on this PR: options are now one paragraph each with the decision and its reasons in the Decision section; the gating no longer lives in GH workflow YAML at all; and no variables pass between invocations, dissolving the portability concern rather than specifying a mechanism for it. The non-template References section is folded into inline links. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
Summary
CLI half of the direction agreed on #4718 (comment, acked by @ifireball, @rh-hemartin, @ggallen): pre-scripts get a first-class skip protocol inside
fullsend run, so the reusable workflows' inline pre-script calls — the source of the double execution, and the last consumer of the scaffold script copies (#5667) — become removable in a follow-up PR.Related Issue
Part of #4718 (also unblocks #5667, closed as duplicate into this effort).
Changes
internal/prescriptpackage owning the protocol:fullsend runexportsFULLSEND_PRESCRIPT_OUTPUT(temp file path) into the pre-script env; the script may append GitHub-output-stylekey=valuelines (skipped=true, optionalreason=...). Missing/empty file → proceed (today's behavior). Malformed content is a hard error so a mistyped skip cannot silently proceed as a duplicate agent run.internal/cli/run.go: pre-script block extracted into arunPreScripthelper wired to the protocol; onskipped=truethe run reports a skipped status, relays outputs toGITHUB_OUTPUT(when under GHA; auto-detected in the CLI, not hard-coded in workflow YAML — portable per the review discussion on docs(#4718): add ADR 0072 for the pre-script output protocol #5016), and exits 0 before sandbox creation.internal/statuscomment: newskippedcompletion status (⏭️).Version skew (ADR 0062): both directions degrade to current behavior — scripts unaware of the protocol proceed as before; protocol-aware scripts must guard on the env var being unset under older CLIs (same pattern as the existing
GITHUB_OUTPUTguard).Follow-ups (sequenced on #4718): workflow cleanup + scaffold script deletion (resolves #5667), agents-repo script changes (recut of fullsend-ai/agents#175), ADR rewrite (#5016).
Testing
make lintpasses (staged)internal/prescript,runPreScriptexercised with fixture scripts (skip, proceed, runner-env, script failure, malformed output),statuscommentskipped-status testmake go-vet,go build ./...,go test -raceon changed packages; fullgo test ./...run — one pre-existing failure on pristineorigin/mainunrelated to this diff (TestListTriggeredHarnesses_BaseComposition, macOS temp-dir symlink vs workspace-root escape check)make e2e-testnot run locally (requires live pool orgs + mint OIDC); covered by CIChecklist
!for breaking changes)