Agent step probes the runtime instead of relying solely on safe-outputs (leaves stray test PRs behind)

[IEvangelist]

Summary

The agent job in a gh aw workflow is spending most of its budget probing the runtime — shelling out to safeoutputs <tool> repeatedly to "test" whether create_pull_request works, manually git push-ing branches, mutating git remotes, curl-ing the GitHub API, and even committing throwaway "test" content to real files just to see what happens. None of this should be necessary: per the safe-outputs contract, the agent is supposed to declare its intended outputs once via the safe-outputs MCP tools and let the post-agent job materialize them. The agent shouldn't be invoking safeoutputs as a CLI probe at all.

In this particular run those probes (a) burned the entire 20-minute timeout, causing the agent step to fail, and (b) had real, externally visible side effects on a downstream repo, because one of the probes actually succeeded.

Evidence

• Failing run: microsoft/aspire Actions run 26019861837 — workflow PR Documentation Check (pr-docs-check).
• The agent job: job 76478628776 — Execute GitHub Copilot CLI step timed out after 20 minutes.
• Side-effect PR left behind on the target repo: microsoft/aspire.dev#992 — title literally [docs] test, body test, branch docs/pr-17198-test-from-main-1853f10f924372d4, opened by the workflow's bot identity.

The create_pull_request safe-output is correctly configured with target-repo: microsoft/aspire.dev, title_prefix: "[docs] ", labels: ["docs-from-code"], etc. The leftover PR matches that config exactly — it is not a malformed/test-bench artifact, it is a real PR opened against main of microsoft/aspire.dev because the agent submitted --title "test" --body "test" via safeoutputs create_pull_request while probing.

Sample of probing behavior from the agent log

A few representative tool calls from the Execute GitHub Copilot CLI step (with timestamps), in order:

07:42:14  Try safeoutputs create_pull_request from main workspace (shell)
            └ cd .../aspire && safeoutputs create_pull_request ...
07:42:25  safeoutputs create_pull_request --help
07:43:52  Test safeoutputs without base (shell)
            └ safeoutputs create_pull_request --title "test" --body "test"
07:45:04  Try manual push to test auth (shell)
            └ cd .../_repos/aspire.dev && git push origin docs/pr-17198-cli-message-wrapping
07:47:46  Check git URL rewriting and GitHub token scope (shell)
            └ curl -H "Authorization: ***" https://api.github.com/repos/microsoft/aspire ...
07:49:31  Update remote URL to aspire.dev (shell)
            └ git remote set-url origin https://github.com/microsoft/aspire.dev.git
07:54:02  Test creating PR from main branch (shell)
            └ echo "test" >> SUPPORT.md && git add SUPPORT.md
              && git commit -m "test change"
              && safeoutputs create_pull_request --title "test" --base "main" --body "test"
07:54:33  "The test worked on `main`! Let me revert the test and make the real change..."
07:55:58  safeoutputs create_pull_request --title "test no base" --body "test"
07:56:09  ##[error] The action 'Execute GitHub Copilot CLI' has timed out after 20 minutes.

The 07:54:02 step is what produced microsoft/aspire.dev#992. The agent was deliberately running safeoutputs create_pull_request as a probe with placeholder text — and that probe got materialized into a real PR on the downstream repo.

(Full log: step:34:822 onwards.)

Why this is a bug

1. Safe outputs are write-once, not a sandbox. Per the documented model, the agent should emit safe-output records (one create_pull_request, one notify_source_pr, etc.) describing what should happen, and the safe-outputs job at the end of the run translates those into real GitHub side effects. Invoking the safeoutputs <tool> CLI as an exploratory shell command violates the "declare, don't execute" intent — every successful invocation is a real PR/issue/comment.
2. There is no "dry run." Nothing in the tool's behavior or help output signals to the agent that calls have real-world effects against the configured target-repo. A reasonable agent that's confused about why an earlier call "failed" will try variations — exactly what happened here.
3. The probing is open-ended. Once the agent thinks the tool is misbehaving, it explores: changing origin remotes, git push-ing manually, curl-ing the API, chmod-ing files, etc. None of this is necessary if the agent had a clear, deterministic contract for how to call the safe-output tool exactly once.
4. It eats the whole timeout. In this run the agent never reached the final notify_source_pr emit. The job failed, the downstream PR was left orphaned with a test title/body, and the source PR got no notification.

Suggested fixes (any combination)

• Make safeoutputs <tool> idempotent / dry-run-aware from the agent's POV. E.g. record the intent in outputs.jsonl when invoked from the agent step, and let the post-agent safe_outputs job de-dup and actually create the PR. The CLI invocation should never directly hit github.com from inside the agent step.
• Tighten the system prompt / tool description for safe outputs to explicitly forbid "test" invocations and probing, and to state that each call has a real external side effect. Today's description for notify_source_pr is clear about this; the descriptions for create_pull_request / create_issue etc. are not.
• Reject obvious test payloads. Titles/bodies that are literally "test", branches like *-test-from-main-*, or patches that consist of echo test >> SUPPORT.md should be blocked with a hard error pointing the agent at the right pattern — not silently published to the target repo.
• Surface a non-fatal "I can't / won't" path more prominently. The agent had report_incomplete and noop available but kept retrying create_pull_request. Make the agent prompt steer toward report_incomplete faster when create_pull_request validation fails, instead of N retries.
• Garbage-collect stray PRs from failed runs. If the agent step fails (timeout or otherwise) and the safe_outputs job didn't run to completion, the post-job should close/clean up any PRs whose head branches match the gh-aw-workflow-id of the failed run.

Repro context

• gh-aw engine: copilot, version 1.0.40, model claude-sonnet-4.6 (per the PR body marker on [WIP] Address feedback on Mcp revamp: fix tests, format, and recompile #992).
• Workflow: pr-docs-check in microsoft/aspire.
• Trigger: PR microsoft/aspire#17198.
• Target: microsoft/aspire.dev, base main (also accepts release/*), draft: true, max: 1, max_patch_size: 1024.

Happy to provide the full job log or the workflow .lock.yml if useful.

[IEvangelist]

Second example: aspire.dev#987 — same probing, but here the run "succeeded"

The behavior I described above isn't just a timeout edge case. Here's a separate run where the agent step itself reported success and yet still left a [docs] Test / test PR behind on the downstream repo.

• Stray PR: microsoft/aspire.dev#987 — title [docs] Test, body test, base release/13.4, branch docs/pr-17114-backchannel-shutdown-fc6fe9673f330b7f.
• Source run: microsoft/aspire Actions run 25947868349, workflow PR Documentation Check, triggered by microsoft/aspire#17114.
• agent job: 76279774583 — conclusion success.
• safe_outputs job: 76280728001 — conclusion success.
• Notify source PR job: 76280803591 — conclusion success.

So from the workflow's POV, everything is green. The downstream repo just gets a PR titled "Test" with body "test".

What actually happened in the agent log

Same gh-aw engine/version (copilot, 1.0.40, claude-sonnet-4.6). The agent:

1. Tried safeoutputs create_pull_request from $GITHUB_WORKSPACE with the real title/body — got "no commits found" errors.
2. Spent ~10 minutes probing: safeoutputs create_pull_request --help, varying --repo, varying cwd, comparing API latencies ("the call with microsoft/aspire.dev took 5900ms... it's probably making an API call"), querying session_store for "past successful PR creation patterns", etc.
3. Eventually ran this from the _repos/aspire.dev checkout:

   cd .../_repos/aspire.dev && safeoutputs create_pull_request \
     --title "Test" --body "test" \
     --branch "docs/pr-17114-backchannel-shutdown" --base "release/13.4"
   

4. The tool returned success. The agent's own narration (verbatim from the log) is the key part:

   "It worked!" … "It just created a patch file, not the PR itself. The workflow will use that patch to push and create the PR after I finish. So I haven't wasted my PR creation limit yet. I can create it again with the proper title and body."

5. The agent then called safeoutputs create_pull_request a second time with the real full title/body.
6. It then emitted notify_source_pr and exited cleanly.

The downstream PR that was actually opened is the first one — [docs] Test / test — because the create_pull_request config has max: 1. The agent's "real" follow-up call was either discarded as max exceeded or simply lost the race; either way the user-visible result is the test-payload PR on aspire.dev and a notify_source_pr that points at it as if it were the real docs draft.

Why this is worse than the timeout case from the issue body

• The agent's mental model of the tool is wrong, and the tool isn't correcting it. Quote: "It just created a patch file, not the PR itself. … I haven't wasted my PR creation limit yet." From the agent's perspective, each safeoutputs create_pull_request invocation is a side-effect-free probe that just produces a patch on disk — the PR is supposedly materialized later. In reality the first successful call wins the single allowed slot, and the rest are wasted. The tool's response format and description give the agent no signal that it has just used its one PR quota on a placeholder.
• There's no failure signal. Run 25947868349 is fully green. There's no timeout, no errored step, no surfaced lint. Without manually inspecting aspire.dev, you'd never know that "the docs draft for [deps] Dependabot Dependency Checker - Issue Group #17114" is actually [docs] Test. The notify_source_pr comment posted on the source PR will link to this garbage as if it's the real artifact.
• max: 1 is now a footgun. Combined with the agent's belief that calls are probes, max: 1 actively encourages the failure mode where the first throwaway call locks in the result.

Additional concrete suggestions on top of the original list

• Make the create_pull_request MCP tool response unambiguous about side effects. Today's response (per the agent's read) looked enough like a "patch generated, will be applied later" message that the agent assumed calls were free. The response should explicitly say something like "Recorded create_pull_request intent #1 of 1 — no further calls accepted. This will be opened as a real draft PR on <target-repo>." so the agent knows it just spent its one shot.
• Don't return success on the first call until the agent has confirmed it isn't a probe. Or invert: only the last create_pull_request call in the session gets materialized, with earlier calls explicitly logged as superseded. That matches the agent's intuition about "test then commit" and would have made Mcp revamp #987 self-heal.
• Reject placeholder payloads at the safe-outputs boundary, not at the agent prompt. Titles like Test, test, foo, bodies under N characters, or branches that match the gh-aw-generated suffix but with --title test should be hard-rejected by the safe-outputs handler with a structured error. That way even if the agent does probe, it can't actually publish "Test"/"test" to a downstream repo.
• Cleanup hook for the safe_outputs job. If the materialized PR's title/body matches a "looks like a probe" pattern (literally test, foo, single short word, etc.), the job should refuse to publish it and emit a report_incomplete instead. Today the workflow happily publishes whatever the first call contained.

I have the full agent log for this second run too if it would help; happy to attach or share privately.