Skip to content

Add shared safe-output-app workflow for repository-level GitHub App authentication#4516

Merged
pelikhan merged 11 commits intomainfrom
copilot/add-shared-agentic-workflow-again
Nov 22, 2025
Merged

Add shared safe-output-app workflow for repository-level GitHub App authentication#4516
pelikhan merged 11 commits intomainfrom
copilot/add-shared-agentic-workflow-again

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Nov 21, 2025

Completed: Add Shared Agentic Workflow for Safe Output App Setup

Summary

Created a new shared workflow that sets up safe output app configuration using repository-level variables and secrets, and integrated it with the changeset generator workflow.

What was implemented

  1. New Shared Workflow (.github/workflows/shared/safe-output-app.md)

    • Provides repository-level GitHub App configuration for safe outputs
    • Uses APP_ID (repository variable) and APP_PRIVATE_KEY (repository secret)
    • Complements the existing app-config.md which uses organization-level credentials
  2. Changeset Generator Integration

    • Updated .github/workflows/changeset.md to import the new shared workflow
    • The workflow now automatically uses GitHub App authentication when configured
  3. Enhanced push_to_pull_request_branch Support

    • Fixed the push_to_pull_request_branch job to properly support GitHub App tokens
    • Added automatic app token minting step when app configuration is detected
    • Updated GH_TOKEN environment variable to use app token when configured
    • Updated Git configuration to authenticate with app token when configured
  4. Fixed conclusion job

    • Added app token minting step to conclusion job when app is configured
    • Added app token invalidation step to conclusion job when app is configured
    • Ensures noop processing step can access the app token
    • Properly cleans up app tokens after job completion
    • Resolves "Input required and not supplied: github-token" error
  5. Fixed GitHub App Permissions Mapping

    • Updated convertPermissionsToAppTokenFields to only include valid GitHub App permissions
    • Removed invalid permissions: discussions, models, id-token, attestations
    • Added documentation explaining the mapping differences
    • Prevents "Unexpected input" warnings from actions/create-github-app-token
    • Sorted permission keys alphabetically for deterministic compilation order
  6. Auto-set Owner and Repositories

    • When owner is not specified, defaults to ${{ github.repository_owner }}
    • When repositories is not specified, defaults to ${{ github.event.repository.name }}
    • Eliminates the warning: "Inputs 'owner' and 'repositories' are not set"
    • Explicitly specifying these values still works as expected
  7. Updated Action Pin

    • Updated actions/create-github-app-token@v2 SHA to latest version
    • Fixes test failure in TestActionPinSHAsMatchVersionTags
  8. Integrated with Existing Daily Workflows

    • Updated daily-file-diet.md to import shared/safe-output-app.md (creates issues)
    • Updated artifacts-summary.md to import shared/safe-output-app.md (creates discussions)
    • Both workflows now use GitHub App authentication for safe outputs
    • Demonstrates usage pattern with real production workflows
  9. New Helper Function

    • Created generateGitConfigurationStepsWithToken in yaml_generation.go
    • Allows flexible token selection for Git operations
    • Maintains backward compatibility with existing workflows
  10. Changeset Documentation

  • Created .changeset/patch-add-safe-output-app-shared-workflow.md
  • Documents the change as a patch-level update

Testing

  • ✅ All unit tests pass
  • ✅ Changeset workflow compiles successfully
  • ✅ App token minting step is correctly generated in all jobs that need it
  • ✅ App token invalidation step is correctly generated in all jobs that need it
  • ✅ Conclusion job now has both app-token minting and invalidation steps
  • ✅ Workflows without app configuration don't have token minting/invalidation steps
  • ✅ No invalid GitHub App permissions (discussions, models) in compiled workflows
  • ✅ Permission keys are sorted alphabetically for deterministic compilation
  • ✅ Owner and repositories are automatically set when not specified
  • ✅ Explicitly specified owner and repositories are preserved
  • ✅ Daily-file-diet and artifacts-summary workflows compile successfully with shared workflow
  • ✅ Git configuration uses app token when configured
  • ✅ GitHub Script steps receive app token via github-token parameter

Files Changed

  • .github/workflows/shared/safe-output-app.md (new)
  • .github/workflows/changeset.md (updated to import shared workflow)
  • .github/workflows/changeset.lock.yml (recompiled)
  • .github/workflows/daily-file-diet.md (updated to import shared workflow)
  • .github/workflows/daily-file-diet.lock.yml (recompiled)
  • .github/workflows/artifacts-summary.md (updated to import shared workflow)
  • .github/workflows/artifacts-summary.lock.yml (recompiled)
  • .github/aw/actions-lock.json (updated action pin)
  • pkg/workflow/data/action_pins.json (synced from actions-lock.json)
  • pkg/workflow/push_to_pull_request_branch.go (enhanced for app token support)
  • pkg/workflow/notify_comment.go (added app-token minting and invalidation steps to conclusion job)
  • pkg/workflow/safe_outputs_app.go (fixed GitHub App permissions mapping, added deterministic sorting, auto-set owner/repositories)
  • pkg/workflow/yaml_generation.go (added token parameter support)
  • .changeset/patch-add-safe-output-app-shared-workflow.md (new)
Original prompt

Add a shared agentic workflow that sets up the safe output app to

app id is in Repository Variables APP_ID
Private key: secret APP_PRIVATE_KEY

Use shared workflow with changeset generator


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits November 21, 2025 21:47
…rator

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add shared agentic workflow for safe output app Add shared safe-output-app workflow for repository-level GitHub App authentication Nov 21, 2025
Copilot AI requested a review from pelikhan November 21, 2025 21:50
@github-actions
Copy link
Copy Markdown
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot the run failed in the conclusion when processing the noop message. Investigate.

https://github.com/githubnext/gh-aw/actions/runs/19584497219/job/56090336863#step:5:1

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Nov 21, 2025

@copilot the run failed in the conclusion when processing the noop message. Investigate.

https://github.com/githubnext/gh-aw/actions/runs/19584497219/job/56090336863#step:5:1

Fixed in 150a5a6. The issue was that the conclusion job was trying to use ${{ steps.app-token.outputs.token }} but didn't have the app-token minting step. Added the app-token minting step to the beginning of the conclusion job when app configuration is present.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot also add the token invalidation step in the conclusion job. Make sure those steps are added only if safe-outputs.app is defined.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Nov 21, 2025

@copilot also add the token invalidation step in the conclusion job. Make sure those steps are added only if safe-outputs.app is defined.

Added in b19f253. The token invalidation step is now added to the conclusion job when safe-outputs.app is defined, matching the pattern used in other safe output jobs. The step runs with if: always() && steps.app-token.outputs.token != '' to ensure proper cleanup even on failure.

@github-actions
Copy link
Copy Markdown
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot The permissions used by the actions/create-github-app are App permissions, not github action permissions. There are slightly different.

This is the error message when trying permission-discussions which does not exist.

Warning: Unexpected input(s) 'permission-discussions', valid inputs are ['app-id', 'private-key', 'owner', 'repositories', 'skip-token-revoke', 'github-api-url', 'permission-actions', 'permission-administration', 'permission-checks', 'permission-codespaces', 'permission-contents', 'permission-dependabot-secrets', 'permission-deployments', 'permission-email-addresses', 'permission-environments', 'permission-followers', 'permission-git-ssh-keys', 'permission-gpg-keys', 'permission-interaction-limits', 'permission-issues', 'permission-members', 'permission-metadata', 'permission-organization-administration', 'permission-organization-announcement-banners', 'permission-organization-copilot-seat-management', 'permission-organization-custom-org-roles', 'permission-organization-custom-properties', 'permission-organization-custom-roles', 'permission-organization-events', 'permission-organization-hooks', 'permission-organization-packages', 'permission-organization-personal-access-token-requests', 'permission-organization-personal-access-tokens', 'permission-organization-plan', 'permission-organization-projects', 'permission-organization-secrets', 'permission-organization-self-hosted-runners', 'permission-organization-user-blocking', 'permission-packages', 'permission-pages', 'permission-profile', 'permission-pull-requests', 'permission-repository-custom-properties', 'permission-repository-hooks', 'permission-repository-projects', 'permission-secret-scanning-alerts', 'permission-secrets', 'permission-security-events', 'permission-single-file', 'permission-starring', 'permission-statuses', 'permission-team-discussions', 'permission-vulnerability-alerts', 'permission-workflows']

Update the code in the compiler that computes the permission-... object to account for GitHub App permissions, not action persmissions.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Nov 21, 2025

@copilot The permissions used by the actions/create-github-app are App permissions, not github action permissions. There are slightly different.

This is the error message when trying permission-di...

Fixed in 1447bac. Updated convertPermissionsToAppTokenFields to only map GitHub Actions permissions that have corresponding GitHub App permissions. Removed invalid permissions: discussions, models, id-token, attestations, and repository-projects. The function now only includes valid GitHub App permissions: actions, checks, contents, deployments, issues, packages, pages, pull-requests, security-events, and statuses.

Comment thread .github/workflows/changeset.lock.yml Outdated
app-id: ${{ vars.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
github-api-url: ${{ github.api_url }}
permission-contents: read
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot make sure that keys permission-... are stored to ensure deterministic compilation order.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d15e5fd. The permission-* keys are now sorted alphabetically before being added to the workflow YAML, ensuring deterministic compilation order. The fix extracts all keys from the permission fields map, sorts them using sort.Strings(), and then iterates in sorted order when appending to the steps.

@github-actions
Copy link
Copy Markdown
Contributor

Agentic Changeset Generator triggered by this pull request.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot it fails with this log:

2025-11-21T23:31:59.4985854Z ##[group]Run actions/create-github-app-token@6701853
2025-11-21T23:31:59.4988203Z with:
2025-11-21T23:31:59.4988969Z app-id: 2331539
2025-11-21T23:31:59.4990980Z private-key: ***
2025-11-21T23:31:59.4991787Z owner: githubnext
2025-11-21T23:31:59.4992554Z repositories: gh-aw
2025-11-21T23:31:59.4993432Z github-api-url: https://api.github.com
2025-11-21T23:31:59.4994474Z permission-contents: read
2025-11-21T23:31:59.4995362Z permission-issues: write
2025-11-21T23:31:59.4996261Z permission-pull-requests: write
2025-11-21T23:31:59.4997467Z skip-token-revoke: false
2025-11-21T23:31:59.4998643Z ##[endgroup]
2025-11-21T23:31:59.9800325Z Inputs 'owner' and 'repositories' are set. Creating token for the following repositories:
2025-11-21T23:31:59.9803438Z
2025-11-21T23:31:59.9804667Z - githubnext/gh-aw
2025-11-21T23:31:59.9870845Z Failed to create token for "gh-aw" (attempt 1): Invalid keyData
2025-11-21T23:31:59.9923802Z DOMException [DataError]: Invalid keyData
2025-11-21T23:31:59.9929968Z at Object.rsaImportKey (node:internal/crypto/rsa:235:15)
2025-11-21T23:31:59.9933231Z at SubtleCrypto.importKey (node:internal/crypto/webcrypto:616:10)
2025-11-21T23:31:59.9935631Z ... 6 lines matching cause stack trace ...
2025-11-21T23:31:59.9941197Z at pRetry.shouldRetry (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:44597:13)
2025-11-21T23:31:59.9947966Z at RetryOperation._fn (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:44521:30) {
2025-11-21T23:31:59.9953039Z attemptNumber: 1,
2025-11-21T23:31:59.9962839Z retriesLeft: 3,
2025-11-21T23:31:59.9964076Z [cause]: Error: Failed to read private key
2025-11-21T23:31:59.9965346Z at createPrivateKey (node:internal/crypto/keys:631:12)
2025-11-21T23:31:59.9969360Z at Object.rsaImportKey (node:internal/crypto/rsa:229:21)
2025-11-21T23:31:59.9971069Z at SubtleCrypto.importKey (node:internal/crypto/webcrypto:616:10)
2025-11-21T23:31:59.9978439Z at getToken (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:43791:56)
2025-11-21T23:31:59.9991515Z at githubAppJwt (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:43824:23)
2025-11-21T23:31:59.9999537Z at getAppAuthentication (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:43997:37)
2025-11-21T23:32:00.0005502Z at hook4 (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:44309:37)
2025-11-21T23:32:00.0012587Z at newApi (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:43100:36)
2025-11-21T23:32:00.0017976Z at getTokenFromRepository (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:44655:26)
2025-11-21T23:32:00.0027090Z at pRetry.shouldRetry (/home/runner/work/_actions/actions/create-github-app-token/67018539274d69449ef7c02e8e71183d1719ab42/dist/main.cjs:44597:13)
2025-11-21T23:32:00.0036957Z }
2025-11-21T23:32:00.0118259Z ##[error]Invalid keyData

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Nov 21, 2025

@copilot it fails with this log:

...

This error indicates the APP_PRIVATE_KEY secret is not in the correct format. The GitHub App private key must be in PEM format (the full content from the downloaded .pem file, including the -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY----- markers).

To fix:

  1. Go to your GitHub App settings
  2. Generate/download the private key (it will be a .pem file)
  3. Copy the entire contents of the file
  4. Update the APP_PRIVATE_KEY repository secret with this content

The compiler code is working correctly - the expressions are being rendered properly in the YAML. The issue is with the actual secret value stored in the repository settings.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 22, 2025

✅ Agentic Changeset Generator completed successfully.

@pelikhan
Copy link
Copy Markdown
Collaborator

@Format lint, fix tests

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot add import of the shared app shared agentic workflow to a daily agentic workflow that creates an issue and a daily agentic workflow that creates a discussion.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Nov 22, 2025

@copilot add import of the shared app shared agentic workflow to a daily agentic workflow that creates an issue and a daily agentic workflow that creates a discussion.

Added in cf4acbc. Created two example daily workflows:

  1. daily-issue-example.md - Scheduled daily workflow that creates an issue using GitHub App authentication
  2. daily-discussion-example.md - Scheduled daily workflow that creates a discussion using GitHub App authentication

Both workflows import shared/safe-output-app.md for authentication and demonstrate the usage pattern. They compile successfully and include:

  • GitHub App token minting with auto-configured owner and repositories
  • Proper permissions for their respective safe outputs
  • Token invalidation for cleanup

Also updated the actions/create-github-app-token@v2 action pin to the latest SHA, fixing the test failure.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot revert the privous commit. Instead of creating 2 new agentic workflows, update 2 existing agentic workflows in the .github/workflows folder and import the app shared agentic workflow in those.

…g new ones

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Nov 22, 2025

@copilot revert the privous commit. Instead of creating 2 new agentic workflows, update 2 existing agentic workflows in the .github/workflows folder and import the app shared agentic workflow in those...

Fixed in 6bff113. Removed the newly created example workflows and instead updated two existing daily workflows:

  1. daily-file-diet.md - Added import of shared/safe-output-app.md (creates issues)
  2. artifacts-summary.md - Added import of shared/safe-output-app.md (creates discussions)

Both workflows now use GitHub App authentication for their safe outputs. The workflows compile successfully and include app token minting/invalidation steps.

@pelikhan pelikhan marked this pull request as ready for review November 22, 2025 01:11
Copilot AI review requested due to automatic review settings November 22, 2025 01:11
@pelikhan pelikhan merged commit 1c784e9 into main Nov 22, 2025
54 of 55 checks passed
@pelikhan pelikhan deleted the copilot/add-shared-agentic-workflow-again branch November 22, 2025 01:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds repository-level GitHub App authentication support for safe-output operations, introducing a shared workflow configuration (shared/safe-output-app.md) that can be imported by workflows needing GitHub App tokens. The implementation includes automatic token minting/invalidation, default owner/repositories values, deterministic permission sorting, and integration with git operations and GitHub CLI commands.

Key changes:

  • New shared workflow for centralized GitHub App configuration using repository variables/secrets
  • Enhanced safe-output jobs to support GitHub App token authentication
  • Fixed permission mapping to exclude invalid GitHub App permissions
  • Auto-set owner and repositories when not explicitly configured

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
.github/workflows/shared/safe-output-app.md New shared workflow providing repository-level GitHub App credentials configuration
pkg/workflow/safe_outputs_app.go Fixed permissions mapping to exclude discussions/models, added deterministic sorting, auto-set defaults
pkg/workflow/yaml_generation.go Added helper function for git configuration with custom tokens
pkg/workflow/push_to_pull_request_branch.go Integrated app token support for git operations and GH CLI
pkg/workflow/notify_comment.go Added app token minting/invalidation to conclusion job
.github/workflows/changeset.md Imported shared safe-output-app workflow
.github/workflows/daily-file-diet.md Imported shared safe-output-app workflow
.github/workflows/artifacts-summary.md Imported shared safe-output-app workflow
.github/aw/actions-lock.json Updated action pin to latest SHA
pkg/workflow/data/action_pins.json Synced with actions-lock.json
.changeset/patch-add-safe-output-app-shared-workflow.md Changeset documenting patch-level update

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- **Centralized Configuration**: Single source of truth for app credentials
- **Easy Updates**: Change credentials in one place
- **Consistent Usage**: All workflows use the same configuration pattern
- **Repository-Scoped**: Uses repository-specific variables and secrets
Copy link

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared workflow documentation should mention the limitations of GitHub App authentication. Specifically, GitHub Apps don't support the discussions permission scope, which means workflows using safe-outputs.create-discussion will fail when combined with safe-outputs.app.

Suggested addition to the documentation (after line 29):

## Limitations

- **Discussions Not Supported**: GitHub Apps don't support the `discussions` permission. Workflows using `safe-outputs.create-discussion` should not import this shared workflow, as discussion creation will fail with authorization errors.

This will help users understand which safe-output types are compatible with GitHub App authentication.

Suggested change
- **Repository-Scoped**: Uses repository-specific variables and secrets
- **Repository-Scoped**: Uses repository-specific variables and secrets
## Limitations
- **Discussions Not Supported**: GitHub Apps don't support the `discussions` permission. Workflows using `safe-outputs.create-discussion` should not import this shared workflow, as discussion creation will fail with authorization errors.

Copilot uses AI. Check for mistakes.
Comment on lines +4497 to 4507
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@7e473efe3cb98aa54f8d4bac15400b15fad77d94 # v2
with:
app-id: ${{ vars.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: ${{ github.event.repository.name }}
github-api-url: ${{ github.api_url }}
permission-contents: read
- name: Download agent output artifact
Copy link

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The create_discussion job may fail at runtime when using GitHub App authentication. The job requires discussions: write permission to create discussions, but GitHub Apps don't support this permission scope (as documented in lines 209-214).

When the app token is minted for this job (line 4506 in artifacts-summary.lock.yml), it only receives permission-contents: read without any discussions permission. This means any attempt to create a discussion using the app token will fail with an authorization error.

Impact: The create_discussion safe output will not work when safe-outputs.app is configured.

Suggested fix: Either:

  1. Validate at compile time that create-discussion and safe-outputs.app cannot be used together, OR
  2. Fall back to using GITHUB_TOKEN instead of the app token for discussion creation, OR
  3. Document this limitation prominently in the shared workflow documentation

Copilot uses AI. Check for mistakes.
steps:
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2
Copy link

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test workflow file is still using the old SHA for actions/create-github-app-token@v2 (67018539274d69449ef7c02e8e71183d1719ab42), while the action pin has been updated to 7e473efe3cb98aa54f8d4bac15400b15fad77d94 in both actions-lock.json and action_pins.json.

This suggests the test workflow needs to be recompiled to pick up the updated action pin. All test workflows should use the same pinned versions as production workflows for consistency and to ensure tests validate the actual deployed configuration.

Suggested fix: Recompile the test workflow with gh aw compile to update the action pin.

Suggested change
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2
uses: actions/create-github-app-token@7e473efe3cb98aa54f8d4bac15400b15fad77d94 # v2

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants