Skip to content

Add noop safe output for transparent workflow completion#4286

Merged
pelikhan merged 9 commits intomainfrom
copilot/add-no-op-safe-output
Nov 18, 2025
Merged

Add noop safe output for transparent workflow completion#4286
pelikhan merged 9 commits intomainfrom
copilot/add-no-op-safe-output

Conversation

Copy link
Contributor

Copilot AI commented Nov 18, 2025

Agents need to emit human-visible artifacts even when no actions are required (e.g., "No issues found"). The noop safe output provides a fallback mechanism ensuring workflows never complete silently.

Changes

Schema & Types

  • Added NoOpOutput schema definition with required message field
  • Added TypeScript interfaces (NoOpItem, NoOpConfig) for type safety
  • Updated workflow schema to accept noop configuration

JavaScript Implementation

  • collect_ndjson_output.cjs: Validation case with message sanitization (neutralizes @mentions, etc.)
  • notify_comment_error.cjs: Integrated noop message display into conclusion job with dual-mode handling:
    • When activation comment exists: appends noop messages to the comment
    • When no activation comment: writes noop messages to step summary
  • Bundling Fix: Moved notify_comment_error.cjs from js.go to scripts.go to enable proper bundling of load_agent_output.cjs dependency. The bundler now inlines all required modules directly into compiled workflows, eliminating runtime require() errors.
  • Default max: 1 message (consistent with other safe outputs)

Go Integration

  • noop.go: Configuration parser with default max of 1
  • safe_outputs.go: Auto-enable logic - noop enabled by default when safe-outputs exists; added noop to tool filtering logic for MCP integration
  • notify_comment.go: Updated to always generate conclusion job when safe-outputs exist (not just when activation comments are present)
  • scripts.go: Added getNotifyCommentErrorScript() bundling function with lazy bundling and caching, following the same pattern as other bundled scripts

MCP Integration

  • safe_outputs_tools.json: Added noop tool definition with proper schema for MCP server integration
  • Tool includes description and input schema (message field required)
  • Enables AI agents to discover and use the noop tool through the safe outputs MCP server
  • Updated test expectations to include noop in the tool list

Architecture

Noop messages are processed in the conclusion job (not as a separate job). The conclusion job is now always generated when safe-outputs exist, ensuring noop messages can be handled regardless of whether activation comments are configured.

Behavior:

  • With activation comment (reactions configured): Noop messages are appended to the completion status comment
  • Without activation comment: Noop messages are written to the step summary for transparency

This guarantees that noop messages are always visible somewhere, preventing silent workflow completion.

Bundling Implementation

The notify_comment_error.cjs script uses require("./load_agent_output.cjs") to load agent output artifacts. To ensure this works in GitHub Actions runtime:

  1. Script moved to scripts.go (bundled scripts) from js.go (standalone scripts)
  2. Bundler automatically inlines load_agent_output.cjs content directly into the compiled workflow
  3. No external require() calls remain in generated YAML files
  4. All 80 workflows compile successfully with dependencies properly inlined

Usage

---
safe-outputs:
  noop:  # Optional, enabled by default with max: 1
    max: 1
---

# Output format
{"type": "noop", "message": "Analysis complete - no issues found"}

Noop is always available as a fallback and integrates with the conclusion job to ensure human-visible artifacts are produced in all scenarios. The tool is automatically exposed through the safe outputs MCP server for AI agent discovery.

Custom agent used: copilot-add-safe-output-type
Adding a New Safe Output Type to GitHub Agentic Workflows

Custom agent used: copilot-add-safe-output-type
Adding a New Safe Output Type to GitHub Agentic Workflows

Original prompt

Add a "no-op" safe output (always enabled).

Fallback and No-Op Outputs: Finally, to improve safety and clarity, the instructions could define a “no-operation” or fallback safe output. This would be used when the agent decides to take no significant action but still needs to produce a human-visible artifact. For example, a log-only or noop safe output might simply post a comment or discussion stating the agent’s analysis results or that no changes were needed. This addresses scenarios where the agent’s task is informational or where it cannot proceed – instead of failing silently or producing nothing (which, as observed, can confuse the workflow)  , the agent would still output something for transparency. Given the project’s emphasis that agent execution “always leave human‑visible artifacts (comments, PRs, logs) instead of silent mutation” , formalizing a fallback safe output ensures that even if no issue/PR is created, the run isn’t silent. For instance, if an agent finds no problems in code, it could use this output to comment “No issues found in this review” as a record of completion

Takes a message string argument.

The no-op text is used when update the add-comment comment in the default reporting.

Custom agent used: copilot-add-safe-output-type
Adding a New Safe Output Type to GitHub Agentic Workflows


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits November 18, 2025 13:43
…validation

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add no-op safe output for transparency Add noop safe output for transparent workflow completion Nov 18, 2025
Copilot AI requested a review from pelikhan November 18, 2025 13:58
@pelikhan
Copy link
Contributor

@copilot default max is 1

Copy link
Contributor

@pelikhan pelikhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot do not generate a separate noop job, do the no-op processing in the conclusion job instead.

Post the no-op messages to the activation comment if any.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot continue work

Copy link
Contributor Author

Copilot AI commented Nov 18, 2025

@pelikhan Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 04bf955a-aef5-4302-b510-a9488eae60d2

Sorry for the inconvenience!

@pelikhan
Copy link
Contributor

@copilot max should be 1

…ob and fix test workflows

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 18, 2025

@copilot continue work

Continued the work and completed the implementation in commit 0330244. All requested changes have been addressed:

  1. Default max changed to 1 ✓
  2. Separate noop job removed ✓
  3. Noop processing integrated into conclusion job ✓
  4. Noop messages posted to activation comments ✓

The conclusion job now loads agent output and appends noop messages to the completion status comment when present.

@copilot max should be 1

Fixed in commit 0330244. Default max for noop is now 1 across all files:

  • noop.go: parseNoOpConfig default = 1
  • safe_outputs.go: auto-enable default = 1
  • collect_ndjson_output.cjs: getDefaultMaxForType returns 1
  • main_workflow_schema.json: schema default = 1

@github-actions
Copy link
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan
Copy link
Contributor

@copilot the conclusion job should always be generated so that no-ops can be handled

…handle noop messages

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 18, 2025

@copilot the conclusion job should always be generated so that no-ops can be handled

Fixed in commit 34bfcca. The conclusion job is now always generated when safe-outputs exist (because noop is auto-enabled as a fallback).

Key changes:

  • notify_comment.go: Removed reaction requirement - conclusion job created whenever safe-outputs exist
  • notify_comment_error.cjs: Handles missing activation comments gracefully by writing noop messages to step summary
  • Job condition simplified to: (always()) && (needs.agent.result != 'skipped')

Behavior:

  • With activation comment: Updates comment with noop messages
  • Without activation comment: Writes noop messages to step summary for transparency

All tests pass and 80/80 workflows compile successfully.

@pelikhan pelikhan added smoke and removed smoke labels Nov 18, 2025
@github-actions
Copy link
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan
Copy link
Contributor

@copilot make sure load_agent_output.cjs is inlined by the bundler

See error trace at https://github.com/githubnext/gh-aw/actions/runs/19478013400/job/55742960695#step:5:1

Copilot stopped work on behalf of pelikhan due to an error November 18, 2025 19:10
@pelikhan
Copy link
Contributor

@copilot make sure load_agent_output.cjs is inlined by the bundler

See error trace at https://github.com/githubnext/gh-aw/actions/runs/19478013400/job/55742960695#step:5:1

…ript

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot make sure the no-op tool is defined in the safe output tools.JSON

Copy link
Contributor Author

Copilot AI commented Nov 18, 2025

@copilot make sure load_agent_output.cjs is inlined by the bundler

See error trace at https://github.com/githubnext/gh-aw/actions/runs/19478013400/job/55742960695#step:5:1

Fixed in commit 1934f80.

Changes:

  • Moved notify_comment_error.cjs from js.go to scripts.go
  • Added getNotifyCommentErrorScript() bundling function
  • Updated notify_comment.go to call the bundled version

Verification:

  • grep -r "require.*load_agent_output" .github/workflows/*.lock.yml returns 0 results (no require statements)
  • grep -r "function loadAgentOutput" .github/workflows/*.lock.yml returns 153 instances (properly inlined)
  • All 80 workflows compile successfully

The bundler now automatically inlines load_agent_output.cjs into notify_comment_error.cjs, eliminating the require() error.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 18, 2025

@copilot make sure the no-op tool is defined in the safe output tools.JSON

Fixed in commit 3901953.

Changes:

  • Added noop tool definition to safe_outputs_tools.json with proper schema
  • Updated filtering logic in safe_outputs.go to include noop when configured
  • Updated test expectations to include noop in the expected tools list

Verification:

  • All 80 workflows compile successfully with noop tool included
  • Tool definition is properly exposed through the safe outputs MCP server interface
  • Unit tests pass

@github-actions
Copy link
Contributor

github-actions bot commented Nov 18, 2025

✅ Agentic Changeset Generator completed successfully.

@pelikhan pelikhan marked this pull request as ready for review November 18, 2025 21:03
Copilot AI review requested due to automatic review settings November 18, 2025 21:03
@pelikhan pelikhan merged commit 7f1c1ab into main Nov 18, 2025
118 of 120 checks passed
@pelikhan pelikhan deleted the copilot/add-no-op-safe-output branch November 18, 2025 21:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a new "noop" (no-operation) safe output type that allows AI agents to produce human-visible artifacts when no significant actions are required, ensuring workflows never complete silently. The noop output is automatically enabled by default when safe-outputs are configured, providing a transparent fallback mechanism.

Key Changes

  • Added noop schema definition with required message field and comprehensive validation
  • Implemented noop processing in conclusion job with dual-mode support (activation comment vs step summary)
  • Integrated noop tool into MCP server for AI agent discoverability
  • Updated conclusion job logic to always run when safe-outputs exist (not just when activation comments are configured)

Reviewed Changes

Copilot reviewed 93 out of 95 changed files in this pull request and generated no comments.

Show a summary per file
File Description
schemas/agent-output.json Added NoOpOutput schema definition with required message field
pkg/workflow/scripts.go Added bundling support for notify_comment_error script to inline load_agent_output dependency
pkg/workflow/safe_outputs_tools_test.go Updated test expectations to include noop in tool list
pkg/workflow/safe_outputs.go Added auto-enable logic for noop (enabled by default when safe-outputs exist) and MCP tool filtering
pkg/workflow/reaction_none_test.go Updated test to expect conclusion job creation when safe-outputs exist
pkg/workflow/notify_comment_test.go Updated tests to reflect that conclusion job is always created when safe-outputs exist
pkg/workflow/notify_comment.go Changed conclusion job creation logic to always run when safe-outputs exist (not requiring activation comment)
pkg/workflow/noop.go Implemented noop configuration parser with default max of 1
pkg/workflow/js/types/safe-outputs.d.ts Added NoOpItem TypeScript interface
pkg/workflow/js/types/safe-outputs-config.d.ts Added NoOpConfig TypeScript interface
pkg/workflow/js/safe_outputs_tools.json Added noop tool definition for MCP integration
pkg/workflow/js/notify_comment_error.cjs Enhanced to process noop messages with dual-mode handling (comment update or step summary)
pkg/workflow/js/noop.test.cjs Comprehensive test coverage for noop processing including staged mode, sanitization, and multiple messages
pkg/workflow/js/noop.cjs Main noop implementation with message processing and step summary generation
pkg/workflow/js/collect_ndjson_output.test.cjs Added validation tests for noop output including message requirements and sanitization
pkg/workflow/js/collect_ndjson_output.cjs Added noop validation case with message sanitization and default max of 1
pkg/workflow/js.go Added embedded noop script variable
pkg/workflow/compiler.go Added NoOp field to SafeOutputsConfig struct
pkg/workflow/comment_env_vars_conditional_test.go Updated tests to reflect that conclusion job always has comment env vars when safe-outputs exist
pkg/parser/schemas/main_workflow_schema.json Added noop schema definition with configuration options
pkg/cli/workflows/test-copilot-noop.md Added test workflow for Copilot engine
pkg/cli/workflows/test-codex-noop.md Added test workflow for Codex engine
pkg/cli/workflows/test-claude-noop.md Added test workflow for Claude engine
.github/workflows/*.lock.yml Updated all compiled workflows to include noop tool and conclusion job with bundled load_agent_output
.changeset/patch-add-noop-safe-output.md Added changeset for patch release
Comments suppressed due to low confidence (2)

pkg/workflow/js/notify_comment_error.cjs:1

  • Incorrect property access: loadAgentOutput() returns { success, items } but the code checks for agentOutputResult.data. This should be agentOutputResult.items to match the return structure.
    pkg/workflow/js/notify_comment_error.cjs:1
  • Same incorrect property access pattern: should be agentOutputResult.items instead of agentOutputResult.data to match the return structure from loadAgentOutput().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions bot added a commit that referenced this pull request Nov 19, 2025
Add documentation for three new features merged in the last 24 hours:

1. Update-release safe output (PR #4278)
   - Added to safe outputs table
   - Full documentation with operations (replace, append, prepend)
   - Agent output format examples
   - Configuration options

2. No-op safe output (PR #4286)
   - Added to safe outputs table
   - Documentation explaining transparency and auto-enable
   - Agent output format examples
   - Use cases for silent completion prevention

3. Skip-if-match trigger condition (PR #4287)
   - Added to triggers reference
   - How it works explanation
   - Three common use case examples
   - GitHub search query integration details

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants