-
Notifications
You must be signed in to change notification settings - Fork 393
Add GitHub App installation-token auth for OneLocBuild #17219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
7401e5f
Add GitHub App installation-token auth for OneLocBuild
missymessa e2e3311
Add telemetry categorization and error handling to Get-GitHubAppToken…
missymessa 92cdf50
Merge branch 'main' into missymessa/oneloc-github-app-token
missymessa 2901058
Harden installation lookup: select first match and use truthy check
missymessa f62fc2b
Document OneLocBuild GitHub App onboarding
missymessa 33539ff
Clarify that the shared OneLoc PAT will be retired after App verifica…
missymessa 8eb5e58
Address OneLoc GitHub App review feedback
missymessa 49d3cf9
Merge branch 'main' into missymessa/oneloc-github-app-token
missymessa 4e0c087
Merge branch 'main' into missymessa/oneloc-github-app-token
missymessa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| # Authenticating OneLocBuild's GitHub check-in with the GitHub App | ||
|
|
||
| This document explains how a repository gains access to, and opts in to, the **GitHub App** | ||
| authentication path for the OneLocBuild localization check-in PR. It supplements the main | ||
| [OneLocBuild in Arcade](OneLocBuild.md) documentation. | ||
|
|
||
| ## Background | ||
|
|
||
| When OneLocBuild is configured for a GitHub-based repo (`RepoType: gitHub`), the task opens or | ||
| updates a pull request to check in localized files. The GitHub App authentication path mints a | ||
| repository-scoped installation token (`ghs_…`) at build time, avoiding a stored GitHub credential. | ||
| [GitHub installation tokens expire after one hour](https://docs.github.com/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app#generating-an-installation-access-token). | ||
|
|
||
| The GitHub App used for this is **`dotnet OneLoc Localization`** (owned by `@dotnet-bot`). Its only | ||
| job is to open/update the localization check-in PR on your repository. | ||
|
|
||
| ## How it works (opt-in, backward-compatible) | ||
|
|
||
| The App path is **opt-in** and does not change behavior for any repo that doesn't configure it. In | ||
| [`onelocbuild.yml`](/eng/common/core-templates/job/onelocbuild.yml), the App token is minted only | ||
| when **all** of the following are true: | ||
|
|
||
| - `UseGitHubAppAuthentication` is `true`, **and** | ||
| - `RepoType` is `gitHub`, **and** | ||
| - the build is running in the **`internal`** Azure DevOps project (the App service connection and | ||
| Key Vault key are scoped to `dnceng/internal`). | ||
|
|
||
| When those hold, the job runs [`get-github-app-token.yml`](/eng/common/templates/steps/get-github-app-token.yml), | ||
| which signs a JWT with the App's RSA key in Key Vault, exchanges it for an installation token, and | ||
| passes that token to the OneLocBuild task via `gitHubPatVariable`. | ||
|
|
||
| If `UseGitHubAppAuthentication` is `false` — or the build runs in any project other than `internal` | ||
| (e.g. `DevDiv`, `public`) — the job uses the existing `GithubPat` parameter. | ||
|
|
||
| ## Gaining access | ||
|
|
||
| "Access" means two separate things, and **both** are required: | ||
|
|
||
| 1. **The App must be installed on the GitHub org/account that owns your target repo, and your | ||
| specific repository must be selected in that installation.** The App can only open a PR against a | ||
| repository it is installed on. This is what actually grants the App permission to your repo. | ||
| 2. **Your pipeline must opt in** by setting `UseGitHubAppAuthentication: true` in the | ||
| `onelocbuild.yml` template call. | ||
|
|
||
| ### Step 1 — Request that your repository be added to the App installation | ||
|
|
||
| The App installation and the backing `dnceng/internal` service connection / Key Vault key are | ||
| managed by the .NET Engineering Services (dnceng) team. To have your repo added: | ||
|
|
||
| 1. Identify the **GitHub org** and **repository** your OneLoc check-in PR targets. For most repos | ||
| this is the value of the `GitHubOrg` parameter (default `dotnet`) and your repo name. If you use | ||
| a mirrored repository, it's the `GitHubOrg`/`MirrorRepo` the PR is opened against — **not** the | ||
| Azure DevOps mirror. | ||
| 2. Reach out to the **First Responders** | ||
| [channel](https://teams.microsoft.com/l/channel/19%3Aafba3d1545dd45d7b79f34c1821f6055%40thread.skype/First%20Responders?groupId=4d73664c-9f2f-450d-82a5-c2f02756606d&tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47) | ||
| and ask them to add your repository to the **`dotnet OneLoc Localization`** GitHub App | ||
| installation for the appropriate org. | ||
| 3. The App must have permission to open pull requests (Contents + Pull requests: read & write) on | ||
| the selected repository. dnceng configures this as part of the installation. | ||
|
|
||
| > **Note:** The App is installed per GitHub organization. If your repo lives in an org where the App | ||
| > is not yet installed, dnceng will need to install and approve it in that org first, which may | ||
| > require an org owner's approval. | ||
|
|
||
| ### Step 2 — Opt your pipeline in | ||
|
|
||
| Once your repo is part of the App installation, add the GitHub App parameters to your OneLocBuild | ||
| template call. For example: | ||
|
|
||
| ```yaml | ||
| - ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main') }}: | ||
| - template: /eng/common/templates/job/onelocbuild.yml | ||
| parameters: | ||
| LclSource: lclFilesfromPackage | ||
| LclPackageId: 'LCL-JUNO-PROD-YOURREPO' | ||
| # Opt in to GitHub App authentication for the check-in PR: | ||
| UseGitHubAppAuthentication: true | ||
| ``` | ||
|
|
||
| The dnceng service connection, App client ID, Key Vault, and key name are centralized as defaults in | ||
| the Arcade template. They can be overridden for separately provisioned infrastructure. | ||
|
|
||
| ### GitHub App parameters | ||
|
|
||
| | **Parameter** | **Default** | **Notes** | | ||
| |:-:|:-:|-| | ||
| | `UseGitHubAppAuthentication` | `false` | Activates the App path for GitHub repos in `dnceng/internal`. | | ||
| | `GitHubAppServiceConnection` | `'dnceng-oneloc-githubapp'` | The Azure DevOps **WIF service connection** whose identity has `Sign` permission on the App's Key Vault key. | | ||
| | `GitHubAppClientId` | `'Iv23lijBU8x3gc9lDOc9'` | The GitHub App's **Client ID** (used as the JWT `iss` claim). | | ||
| | `GitHubAppKeyVaultName` | `'EngKeyVault'` | The Key Vault holding the App's RSA signing key. | | ||
| | `GitHubAppKeyName` | `'oneloc-localization-app-key'` | The name of the RSA key inside that Key Vault (the App's private key). | | ||
|
|
||
| The token is minted for the installation on the `GitHubOrg` account (default `dotnet`), so make sure | ||
| `GitHubOrg` (and `MirrorRepo`, if mirroring) point at the org/repo where the App is installed. | ||
|
|
||
| ## Verifying it works | ||
|
|
||
| 1. Run your pipeline (on the `internal` project) from a branch where the OneLocBuild job runs. | ||
| 2. In the build, confirm the **`Get GitHub App installation token`** step runs and succeeds before | ||
| the `OneLocBuild` task. | ||
| 3. Confirm the check-in PR is opened by the **`dotnet OneLoc Localization`** App (the PR author will | ||
| be the App / its bot identity). | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - **The App-token step is skipped.** The App path only activates when | ||
| `UseGitHubAppAuthentication` is `true`, `RepoType` is `gitHub`, and the build runs in the | ||
| `internal` project. Verify all three. | ||
| - **Token minting fails with a Key Vault authorization error.** The service connection identity | ||
| needs the `Key Vault Crypto User` role (or at least the `Sign` action) on the App's key. Contact | ||
| First Responders. | ||
| - **`404`/`Not Found` when requesting the installation token.** The App is not installed on the | ||
| `GitHubOrg` account, or your repository was not selected in the installation. Complete Step 1. | ||
| - **PR fails to open on your repo.** Ensure the App has `Contents` and `Pull requests` (read & | ||
| write) permission on the selected repository, and that your repo is included in the installation. | ||
|
|
||
| ## Scope and limitations | ||
|
|
||
| - The App path is only available in the **`dnceng/internal`** Azure DevOps project. Pipelines in | ||
| other projects use `GithubPat` and are not covered by the `dnceng-oneloc-githubapp` service | ||
| connection. DevDiv can use the same template path after a DevDiv-scoped service connection and | ||
| signing-key access are provisioned. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,139 @@ | ||
| # Mints a short-lived GitHub App installation access token by signing a JWT | ||
| # with a private key stored in Azure Key Vault (RSA, RS256). The signed JWT is | ||
| # exchanged with the GitHub API for a token scoped to a single installation. | ||
| # | ||
| # Requirements: | ||
| # - A GitHub App whose private key has been uploaded into Key Vault as an RSA | ||
| # key (the PEM converted to a Key Vault *key*, NOT stored as a secret). | ||
| # - The caller (the federated Azure service connection used to run this script) | ||
| # must have the `Key Vault Crypto User` role (or at minimum the `Sign` | ||
| # action) on that key. | ||
| # - The App must be installed on the target organization/account | ||
| # (`InstallationOwner`) with the permissions/repositories it needs. | ||
| # | ||
| # Installation tokens (ghs_*) are exempt from the enterprise classic-PAT | ||
| # lifetime policy, which is why this replaces the long-lived PAT. | ||
|
|
||
| [CmdletBinding()] | ||
| param( | ||
| # Name of the Key Vault that holds the GitHub App's RSA signing key. | ||
| [Parameter(Mandatory = $true)] | ||
| [string] $KeyVaultName, | ||
|
|
||
| # Name of the RSA key inside the Key Vault (the App's private key). | ||
| [Parameter(Mandatory = $true)] | ||
| [string] $KeyName, | ||
|
|
||
| # The GitHub App's Client ID (the value to put in the `iss` JWT claim). | ||
| [Parameter(Mandatory = $true)] | ||
| [string] $AppClientId, | ||
|
|
||
| # Login of the organization or user account whose installation we should | ||
| # mint the token for (e.g. `dotnet`, `microsoft`). | ||
| [Parameter(Mandatory = $true)] | ||
| [string] $InstallationOwner, | ||
|
|
||
| # Optional Azure DevOps pipeline variable name to set with the installation | ||
| # token (marked as a secret). When not specified, the token is written to | ||
| # stdout instead. | ||
| [Parameter(Mandatory = $false)] | ||
| [string] $OutputVariableName | ||
| ) | ||
|
|
||
| $ErrorActionPreference = 'Stop' | ||
| $PSNativeCommandUseErrorActionPreference = $true | ||
|
|
||
| . $PSScriptRoot\pipeline-logging-functions.ps1 | ||
|
|
||
| function ConvertTo-Base64Url([byte[]] $bytes) { | ||
| return [Convert]::ToBase64String($bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_') | ||
| } | ||
|
|
||
| # Build JWT header and payload. Use [ordered] hashtables so JSON | ||
| # serialization is deterministic. | ||
| $jwtHeader = [ordered]@{ | ||
| alg = 'RS256' | ||
| typ = 'JWT' | ||
| } | ||
| $now = [System.DateTimeOffset]::UtcNow | ||
| $jwtPayload = [ordered]@{ | ||
| iat = $now.AddMinutes(-1).ToUnixTimeSeconds() | ||
| exp = $now.AddMinutes(5).ToUnixTimeSeconds() | ||
| iss = $AppClientId | ||
| } | ||
|
|
||
| $headerEncoded = ConvertTo-Base64Url ([System.Text.Encoding]::UTF8.GetBytes(($jwtHeader | ConvertTo-Json -Compress))) | ||
| $payloadEncoded = ConvertTo-Base64Url ([System.Text.Encoding]::UTF8.GetBytes(($jwtPayload | ConvertTo-Json -Compress))) | ||
| $signingInput = "$headerEncoded.$payloadEncoded" | ||
|
|
||
| # Key Vault `sign` expects the *digest* (base64), not the raw bytes. | ||
| $sha256 = [System.Security.Cryptography.SHA256]::Create() | ||
| $digestBytes = $sha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($signingInput)) | ||
| $digestBase64 = [Convert]::ToBase64String($digestBytes) | ||
|
|
||
| Write-Host "Signing JWT with key '$KeyName' in vault '$KeyVaultName'..." | ||
| try { | ||
| $signResponseJson = az keyvault key sign ` | ||
| --vault-name $KeyVaultName ` | ||
| --name $KeyName ` | ||
| --algorithm RS256 ` | ||
| --digest $digestBase64 | ||
| } | ||
| catch { | ||
| Write-PipelineTelemetryError -Category 'Build' -Message "Failed to sign the JWT via Key Vault (key '$KeyName', vault '$KeyVaultName'): $_. Verify the service connection identity has the 'Key Vault Crypto User' role (Sign action) on the key." | ||
| exit 1 | ||
| } | ||
| if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($signResponseJson)) { | ||
| Write-PipelineTelemetryError -Category 'Build' -Message "'az keyvault key sign' exited with code $LASTEXITCODE for key '$KeyName' in vault '$KeyVaultName'. Verify the service connection identity has the 'Key Vault Crypto User' role (Sign action) on the key." | ||
| exit 1 | ||
| } | ||
| $signResponse = $signResponseJson | ConvertFrom-Json | ||
| if ([string]::IsNullOrEmpty($signResponse.signature)) { | ||
| Write-PipelineTelemetryError -Category 'Build' -Message "Key Vault returned an empty signature for key '$KeyName' in vault '$KeyVaultName'." | ||
| exit 1 | ||
| } | ||
| $signatureUrl = $signResponse.signature.TrimEnd('=').Replace('+', '-').Replace('/', '_') | ||
| $jwt = "$signingInput.$signatureUrl" | ||
|
|
||
| $headers = @{ | ||
| Authorization = "Bearer $jwt" | ||
| 'X-GitHub-Api-Version' = '2022-11-28' | ||
| Accept = 'application/vnd.github+json' | ||
| 'User-Agent' = 'dotnet-arcade-onelocbuild' | ||
| } | ||
|
|
||
| Write-Host "Looking up installation for '$InstallationOwner'..." | ||
| try { | ||
| $installations = Invoke-RestMethod -Uri 'https://api.github.com/app/installations' -Headers $headers -Method Get | ||
| } | ||
| catch { | ||
| Write-PipelineTelemetryError -Category 'Build' -Message "Failed to list GitHub App installations: $_. The signed JWT may be invalid or the App's Client ID ('$AppClientId') may be incorrect." | ||
| exit 1 | ||
| } | ||
| $installation = $installations | Where-Object { $_.account.login -eq $InstallationOwner } | Select-Object -First 1 | ||
| if (-not $installation) { | ||
| $found = ($installations | ForEach-Object { $_.account.login }) -join ', ' | ||
| Write-PipelineTelemetryError -Category 'Build' -Message "No installation found for '$InstallationOwner'. App is installed on: $found" | ||
| exit 1 | ||
| } | ||
|
|
||
| try { | ||
| $tokenResponse = Invoke-RestMethod ` | ||
| -Uri "https://api.github.com/app/installations/$($installation.id)/access_tokens" ` | ||
| -Headers $headers ` | ||
| -Method Post ` | ||
| -ContentType 'application/json' | ||
| } | ||
| catch { | ||
| Write-PipelineTelemetryError -Category 'Build' -Message "Failed to mint an installation access token for '$InstallationOwner' (installation $($installation.id)): $_" | ||
| exit 1 | ||
| } | ||
|
|
||
| Write-Host "Got installation token for '$InstallationOwner' (expires $($tokenResponse.expires_at))." | ||
| if ($OutputVariableName) { | ||
| Write-Host "Setting pipeline variable '$OutputVariableName'." | ||
| Write-Host "##vso[task.setvariable variable=$OutputVariableName;issecret=true]$($tokenResponse.token)" | ||
| } | ||
| else { | ||
| Write-Host $tokenResponse.token -ForegroundColor Green | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.