Skip to content

Conclusion comment shows success when safe_outputs fails to submit PR review #30662

@jonathanpeppers

Description

@jonathanpeppers

Summary

When the safe_outputs job fails to submit a PR review (e.g. GitHub API returns 422), the conclusion job still updates the PR comment with "✅ completed successfully!" because it only checks Agent Conclusion and Detection Conclusion, not whether safe outputs were actually delivered.

This makes it look like the reviewer ran and found nothing, when in reality it generated comments that failed to post.

Repro

  1. Trigger an agentic workflow that uses safe-outputs with submit_pull_request_review
  2. If the GitHub API returns a transient 422 error during PR review submission, the safe_outputs job logs ##[error] but the conclusion job still reports success

Example run: https://github.com/dotnet/android/actions/runs/25446317518

Logs from safe_outputs job

Submitting PR review on dotnet/android#11254: event=COMMENT, comments=3, bodyLength=1962
POST /repos/dotnet/android/pulls/11254/reviews - 422 with id 11C0:1C2012:7B8F57:1D76664:69FB6710 in 626ms
##[error]Failed to submit PR review: Unprocessable Entity: "An internal error occurred, please try again."

Logs from conclusion job

Agent Conclusion: success
Detection Conclusion: success
Successfully updated comment

The conclusion job never checks the safe_outputs job result.

Expected behavior

The conclusion comment should show an error (e.g. ❌ or ⚠️) when safe_outputs fails to deliver its outputs, so the user knows to re-trigger the workflow.

Actual behavior

The comment shows "✅ completed successfully!" even though no review comments were posted to the PR.

Metadata

Metadata

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions