Add test agentic workflows for all documented safe output types, fix workflow cancellation behavior, and correct GitHub Actions toolkit usage

[Copilot]

This PR implements comprehensive test workflows for all 10 documented safe output types in the gh-aw system, addressing the need for validation of safe output functionality across different output types, fixes workflow permission check behavior to properly cancel workflows instead of failing them, and corrects incorrect GitHub Actions toolkit API usage.

What was added

Created 10 test agentic workflows in .github/workflows/ following the naming pattern test-safe-output<type>.md:

• create-issue - Tests issue creation with title prefix and labels
• add-issue-comment - Tests commenting on issues and pull requests
• create-pull-request - Tests PR creation with actual file changes
• add-issue-label - Tests label addition with allowlist restrictions
• update-issue - Tests issue updates (title, body, status)
• push-to-pr-branch - Tests pushing changes to PR branches
• missing-tool - Tests missing tool reporting functionality
• create-discussion - Tests repository discussion creation
• create-pull-request-review-comment - Tests line-specific PR review comments
• create-code-scanning-alert - Tests SARIF security finding generation

Key features

Each workflow demonstrates:

• Custom engine implementation that directly appends JSON to the safe output file
• Staged mode (staged: true) to prevent actual GitHub API interactions during testing
• Proper trigger events including workflow_dispatch for manual testing and relevant event types
• Comprehensive test data covering all severity levels, multi-line comments, file changes, etc.
• Safe output validation with proper JSON formatting and required fields

Workflow permission check fixes

Updated JavaScript membership check behavior to properly cancel workflows instead of failing them:

• Fixed check_permissions.cjs: Replaced process.exit() calls with core.setCancelled() for graceful workflow cancellation
• Fixed missing_tool.cjs: Updated error handler to use core.setFailed() instead of process.exit()
• Updated all workflow lock files: Regenerated to include the corrected permission check behavior

GitHub Actions toolkit fixes

Fixed incorrect usage of Actions toolkit APIs as documented in https://github.com/actions/toolkit/tree/main/packages/core:

• Fixed validate_errors.cjs: Replaced core.warn() with correct core.warning() method
• Fixed add_reaction_and_edit_comment.cjs: Replaced core.warn() with correct core.warning() method
• Updated test files: Updated validate_errors.test.cjs to expect the correct core.warning() API calls

This ensures that when permission checks fail (e.g., "Copilot is not a user"), workflows are cancelled gracefully rather than marked as failed, providing better user experience and clearer workflow status. The toolkit fixes ensure proper GitHub Actions integration and logging behavior.

Example usage

The workflows show how custom engines can leverage the safe output system:

# Create an issue via safe output
echo '{"type": "create-issue", "title": "Test Issue", "body": "Issue description", "labels": ["test"]}' >> $GITHUB_AW_SAFE_OUTPUTS

# Add a comment
echo '{"type": "add-issue-comment", "body": "Comment text"}' >> $GITHUB_AW_SAFE_OUTPUTS

Validation

• ✅ All workflows compile successfully with make recompile
• ✅ Generated lock files include proper GITHUB_AW_SAFE_OUTPUTS_STAGED: "true" environment variables
• ✅ Custom engine steps properly format and append JSON to safe output files
• ✅ CLI recognizes all workflows as installed and up-to-date
• ✅ Permission checks now properly cancel workflows instead of failing them
• ✅ All tests pass after permission check fixes
• ✅ All GitHub Actions toolkit APIs now use correct method names
• ✅ JavaScript tests pass with corrected API expectations

This provides a complete reference implementation for each safe output type and enables thorough testing of the safe output processing system, while ensuring proper workflow behavior when permission checks fail and correct GitHub Actions toolkit usage throughout the codebase.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[pelikhan]

@copilot update JavaScript membership check so that it cancels the workflow, not fails it.

Use core.setCancelled to cancel workflow.

See error: Repository permission check failed: Copilot is not a user

[pelikhan]

@copilot update JavaScript membership check so that it cancels the workflow, not fails it.

Use core.setCancelled to cancel workflow.

See error: Repository permission check failed: Copilot is not a user

[Copilot]

@copilot update JavaScript membership check so that it cancels the workflow, not fails it.

Use core.setCancelled to cancel workflow....

Updated JavaScript membership check to use core.setCancelled() instead of process.exit() for cancelling workflows when permission checks fail. The change has been applied to pkg/workflow/js/check_permissions.cjs and regenerated all workflow lock files. See commit 12611ed.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build1334682940/b278/cli.test -test.testlogfile=/tmp/go-build1334682940/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
  • Triggering command: /tmp/go-build743872636/b278/cli.test -test.testlogfile=/tmp/go-build743872636/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

@copilot update JavaScript membership check so that it cancels the workflow, not fails it.

Use core.setCancelled to cancel workflow....

Updated JavaScript membership check to use core.setCancelled() instead of process.exit() for cancelling workflows when permission checks fail. The change has been applied to pkg/workflow/js/check_permissions.cjs and regenerated all workflow lock files. See commit 12611ed.

[Copilot]

@copilot update JavaScript membership check so that it cancels the workflow, not fails it.

Use core.setCancelled to cancel workflow....

Updated the JavaScript membership check to use core.setCancelled() instead of core.error() and process.exit(1) when repository permission checks fail. This addresses the "Copilot is not a user" error by canceling the workflow gracefully instead of failing it. The fix is in commit 087733c.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build1890002915/b278/cli.test -test.testlogfile=/tmp/go-build1890002915/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
  • Triggering command: /tmp/go-build2811146897/b278/cli.test -test.testlogfile=/tmp/go-build2811146897/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

@copilot update JavaScript membership check so that it cancels the workflow, not fails it.

Use core.setCancelled to cancel workflow....

Updated the JavaScript membership check to use core.setCancelled() instead of core.error() and process.exit(1) when repository permission checks fail. This addresses the "Copilot is not a user" error by canceling the workflow gracefully instead of failing it. The fix is in commit 087733c.

[pelikhan]

@copilot review all actions/github-script JavaScript and check for in orrect usage as documented in https://github.com/actions/toolkit/tree/main/packages/core

• search for core.warn(... instead of core.warning(...

[Copilot]

@copilot review all actions/github-script JavaScript and check for in orrect usage as documented in https://github.com/actions/toolkit/tree/main/packages/core

• search for core.warn(... instead of co...

Fixed all incorrect GitHub Actions toolkit usage in JavaScript files. Replaced core.warn() with the correct core.warning() method in validate_errors.cjs and add_reaction_and_edit_comment.cjs, and updated the corresponding test expectations. All JavaScript files now use the proper Actions toolkit API as documented. See commit d799ca6.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build7825527/b278/cli.test -test.testlogfile=/tmp/go-build7825527/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)