diff --git a/Documentation/OneLocBuild.md b/Documentation/OneLocBuild.md index 8ff207681ce..dcf1499a831 100644 --- a/Documentation/OneLocBuild.md +++ b/Documentation/OneLocBuild.md @@ -11,6 +11,11 @@ To make OneLocBuild easier to use, we have integrated the task into Arcade. This To see your repo's current loc configuration, please refer to https://aka.ms/locstats. +> **Authenticating the check-in PR:** GitHub-based repos can use a short-lived, repository-scoped +> **GitHub App** installation token for the localization check-in PR. See +> [Authenticating OneLocBuild's GitHub check-in with the GitHub App](OneLocBuildGitHubApp.md) +> for how to gain access and opt in. + ## Onboarding to OneLocBuild Using Arcade Onboarding to OneLocBuild is a simple process: @@ -195,6 +200,11 @@ The parameters that can be passed to the template are as follows: | `LanguageSet` | `VS_Main_Languages` | This defines the `LanguageSet` of the LocProject.json as described in the [OneLocBuild task documentation](https://dev.azure.com/ceapex/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=languageset%2C-languages-(required)). | | `LclSource` | `LclFilesInRepo` | This passes the `LclSource` input to the OneLocBuild task as described in [its documentation](https://dev.azure.com/ceapex/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=languageset%2C-languages-(required)). For most repos, this should be set to `LclFilesfromPackage`. | | `LclPackageId` | `''` | When `LclSource` is set to `LclFilesfromPackage`, this passes in the package ID as described in the [OneLocBuild task documentation](https://dev.azure.com/ceapex/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=scenario-2%3A-lcl-files-from-a-package). | +| `UseGitHubAppAuthentication` | `false` | Opt in to GitHub App authentication for the check-in PR (`dnceng/internal` + `RepoType: gitHub` only). See [the GitHub App doc](OneLocBuildGitHubApp.md). | +| `GitHubAppServiceConnection` | `'dnceng-oneloc-githubapp'` | The dnceng/internal WIF service connection used to sign the App JWT. Override only when using separately provisioned infrastructure. | +| `GitHubAppClientId` | `'Iv23lijBU8x3gc9lDOc9'` | The GitHub App's Client ID. | +| `GitHubAppKeyVaultName` | `'EngKeyVault'` | Key Vault holding the App's RSA signing key. | +| `GitHubAppKeyName` | `'oneloc-localization-app-key'` | Name of the App's RSA signing key in the Key Vault. | | `condition` | `''` | Allows for conditionalizing the template's steps on build-time variables. | | `JobNameSuffix` | `''` | Allows for custom job name suffix. This is helpful for disambiguation in case of need for more then one OneLocBuild job run - e.g. as a way to set multiple package IDs. | diff --git a/Documentation/OneLocBuildGitHubApp.md b/Documentation/OneLocBuildGitHubApp.md new file mode 100644 index 00000000000..cdbeba3d0ff --- /dev/null +++ b/Documentation/OneLocBuildGitHubApp.md @@ -0,0 +1,122 @@ +# Authenticating OneLocBuild's GitHub check-in with the GitHub App + +This document explains how a repository gains access to, and opts in to, the **GitHub App** +authentication path for the OneLocBuild localization check-in PR. It supplements the main +[OneLocBuild in Arcade](OneLocBuild.md) documentation. + +## Background + +When OneLocBuild is configured for a GitHub-based repo (`RepoType: gitHub`), the task opens or +updates a pull request to check in localized files. The GitHub App authentication path mints a +repository-scoped installation token (`ghs_…`) at build time, avoiding a stored GitHub credential. +[GitHub installation tokens expire after one hour](https://docs.github.com/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app#generating-an-installation-access-token). + +The GitHub App used for this is **`dotnet OneLoc Localization`** (owned by `@dotnet-bot`). Its only +job is to open/update the localization check-in PR on your repository. + +## How it works (opt-in, backward-compatible) + +The App path is **opt-in** and does not change behavior for any repo that doesn't configure it. In +[`onelocbuild.yml`](/eng/common/core-templates/job/onelocbuild.yml), the App token is minted only +when **all** of the following are true: + +- `UseGitHubAppAuthentication` is `true`, **and** +- `RepoType` is `gitHub`, **and** +- the build is running in the **`internal`** Azure DevOps project (the App service connection and + Key Vault key are scoped to `dnceng/internal`). + +When those hold, the job runs [`get-github-app-token.yml`](/eng/common/templates/steps/get-github-app-token.yml), +which signs a JWT with the App's RSA key in Key Vault, exchanges it for an installation token, and +passes that token to the OneLocBuild task via `gitHubPatVariable`. + +If `UseGitHubAppAuthentication` is `false` — or the build runs in any project other than `internal` +(e.g. `DevDiv`, `public`) — the job uses the existing `GithubPat` parameter. + +## Gaining access + +"Access" means two separate things, and **both** are required: + +1. **The App must be installed on the GitHub org/account that owns your target repo, and your + specific repository must be selected in that installation.** The App can only open a PR against a + repository it is installed on. This is what actually grants the App permission to your repo. +2. **Your pipeline must opt in** by setting `UseGitHubAppAuthentication: true` in the + `onelocbuild.yml` template call. + +### Step 1 — Request that your repository be added to the App installation + +The App installation and the backing `dnceng/internal` service connection / Key Vault key are +managed by the .NET Engineering Services (dnceng) team. To have your repo added: + +1. Identify the **GitHub org** and **repository** your OneLoc check-in PR targets. For most repos + this is the value of the `GitHubOrg` parameter (default `dotnet`) and your repo name. If you use + a mirrored repository, it's the `GitHubOrg`/`MirrorRepo` the PR is opened against — **not** the + Azure DevOps mirror. +2. Reach out to the **First Responders** + [channel](https://teams.microsoft.com/l/channel/19%3Aafba3d1545dd45d7b79f34c1821f6055%40thread.skype/First%20Responders?groupId=4d73664c-9f2f-450d-82a5-c2f02756606d&tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47) + and ask them to add your repository to the **`dotnet OneLoc Localization`** GitHub App + installation for the appropriate org. +3. The App must have permission to open pull requests (Contents + Pull requests: read & write) on + the selected repository. dnceng configures this as part of the installation. + +> **Note:** The App is installed per GitHub organization. If your repo lives in an org where the App +> is not yet installed, dnceng will need to install and approve it in that org first, which may +> require an org owner's approval. + +### Step 2 — Opt your pipeline in + +Once your repo is part of the App installation, add the GitHub App parameters to your OneLocBuild +template call. For example: + +```yaml +- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main') }}: + - template: /eng/common/templates/job/onelocbuild.yml + parameters: + LclSource: lclFilesfromPackage + LclPackageId: 'LCL-JUNO-PROD-YOURREPO' + # Opt in to GitHub App authentication for the check-in PR: + UseGitHubAppAuthentication: true +``` + +The dnceng service connection, App client ID, Key Vault, and key name are centralized as defaults in +the Arcade template. They can be overridden for separately provisioned infrastructure. + +### GitHub App parameters + +| **Parameter** | **Default** | **Notes** | +|:-:|:-:|-| +| `UseGitHubAppAuthentication` | `false` | Activates the App path for GitHub repos in `dnceng/internal`. | +| `GitHubAppServiceConnection` | `'dnceng-oneloc-githubapp'` | The Azure DevOps **WIF service connection** whose identity has `Sign` permission on the App's Key Vault key. | +| `GitHubAppClientId` | `'Iv23lijBU8x3gc9lDOc9'` | The GitHub App's **Client ID** (used as the JWT `iss` claim). | +| `GitHubAppKeyVaultName` | `'EngKeyVault'` | The Key Vault holding the App's RSA signing key. | +| `GitHubAppKeyName` | `'oneloc-localization-app-key'` | The name of the RSA key inside that Key Vault (the App's private key). | + +The token is minted for the installation on the `GitHubOrg` account (default `dotnet`), so make sure +`GitHubOrg` (and `MirrorRepo`, if mirroring) point at the org/repo where the App is installed. + +## Verifying it works + +1. Run your pipeline (on the `internal` project) from a branch where the OneLocBuild job runs. +2. In the build, confirm the **`Get GitHub App installation token`** step runs and succeeds before + the `OneLocBuild` task. +3. Confirm the check-in PR is opened by the **`dotnet OneLoc Localization`** App (the PR author will + be the App / its bot identity). + +## Troubleshooting + +- **The App-token step is skipped.** The App path only activates when + `UseGitHubAppAuthentication` is `true`, `RepoType` is `gitHub`, and the build runs in the + `internal` project. Verify all three. +- **Token minting fails with a Key Vault authorization error.** The service connection identity + needs the `Key Vault Crypto User` role (or at least the `Sign` action) on the App's key. Contact + First Responders. +- **`404`/`Not Found` when requesting the installation token.** The App is not installed on the + `GitHubOrg` account, or your repository was not selected in the installation. Complete Step 1. +- **PR fails to open on your repo.** Ensure the App has `Contents` and `Pull requests` (read & + write) permission on the selected repository, and that your repo is included in the installation. + +## Scope and limitations + +- The App path is only available in the **`dnceng/internal`** Azure DevOps project. Pipelines in + other projects use `GithubPat` and are not covered by the `dnceng-oneloc-githubapp` service + connection. DevDiv can use the same template path after a DevDiv-scoped service connection and + signing-key access are provisioned. diff --git a/eng/common/Get-GitHubAppToken.ps1 b/eng/common/Get-GitHubAppToken.ps1 new file mode 100644 index 00000000000..84d70f3ba1f --- /dev/null +++ b/eng/common/Get-GitHubAppToken.ps1 @@ -0,0 +1,139 @@ +# Mints a short-lived GitHub App installation access token by signing a JWT +# with a private key stored in Azure Key Vault (RSA, RS256). The signed JWT is +# exchanged with the GitHub API for a token scoped to a single installation. +# +# Requirements: +# - A GitHub App whose private key has been uploaded into Key Vault as an RSA +# key (the PEM converted to a Key Vault *key*, NOT stored as a secret). +# - The caller (the federated Azure service connection used to run this script) +# must have the `Key Vault Crypto User` role (or at minimum the `Sign` +# action) on that key. +# - The App must be installed on the target organization/account +# (`InstallationOwner`) with the permissions/repositories it needs. +# +# Installation tokens (ghs_*) are exempt from the enterprise classic-PAT +# lifetime policy, which is why this replaces the long-lived PAT. + +[CmdletBinding()] +param( + # Name of the Key Vault that holds the GitHub App's RSA signing key. + [Parameter(Mandatory = $true)] + [string] $KeyVaultName, + + # Name of the RSA key inside the Key Vault (the App's private key). + [Parameter(Mandatory = $true)] + [string] $KeyName, + + # The GitHub App's Client ID (the value to put in the `iss` JWT claim). + [Parameter(Mandatory = $true)] + [string] $AppClientId, + + # Login of the organization or user account whose installation we should + # mint the token for (e.g. `dotnet`, `microsoft`). + [Parameter(Mandatory = $true)] + [string] $InstallationOwner, + + # Optional Azure DevOps pipeline variable name to set with the installation + # token (marked as a secret). When not specified, the token is written to + # stdout instead. + [Parameter(Mandatory = $false)] + [string] $OutputVariableName +) + +$ErrorActionPreference = 'Stop' +$PSNativeCommandUseErrorActionPreference = $true + +. $PSScriptRoot\pipeline-logging-functions.ps1 + +function ConvertTo-Base64Url([byte[]] $bytes) { + return [Convert]::ToBase64String($bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_') +} + +# Build JWT header and payload. Use [ordered] hashtables so JSON +# serialization is deterministic. +$jwtHeader = [ordered]@{ + alg = 'RS256' + typ = 'JWT' +} +$now = [System.DateTimeOffset]::UtcNow +$jwtPayload = [ordered]@{ + iat = $now.AddMinutes(-1).ToUnixTimeSeconds() + exp = $now.AddMinutes(5).ToUnixTimeSeconds() + iss = $AppClientId +} + +$headerEncoded = ConvertTo-Base64Url ([System.Text.Encoding]::UTF8.GetBytes(($jwtHeader | ConvertTo-Json -Compress))) +$payloadEncoded = ConvertTo-Base64Url ([System.Text.Encoding]::UTF8.GetBytes(($jwtPayload | ConvertTo-Json -Compress))) +$signingInput = "$headerEncoded.$payloadEncoded" + +# Key Vault `sign` expects the *digest* (base64), not the raw bytes. +$sha256 = [System.Security.Cryptography.SHA256]::Create() +$digestBytes = $sha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($signingInput)) +$digestBase64 = [Convert]::ToBase64String($digestBytes) + +Write-Host "Signing JWT with key '$KeyName' in vault '$KeyVaultName'..." +try { + $signResponseJson = az keyvault key sign ` + --vault-name $KeyVaultName ` + --name $KeyName ` + --algorithm RS256 ` + --digest $digestBase64 +} +catch { + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to sign the JWT via Key Vault (key '$KeyName', vault '$KeyVaultName'): $_. Verify the service connection identity has the 'Key Vault Crypto User' role (Sign action) on the key." + exit 1 +} +if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($signResponseJson)) { + Write-PipelineTelemetryError -Category 'Build' -Message "'az keyvault key sign' exited with code $LASTEXITCODE for key '$KeyName' in vault '$KeyVaultName'. Verify the service connection identity has the 'Key Vault Crypto User' role (Sign action) on the key." + exit 1 +} +$signResponse = $signResponseJson | ConvertFrom-Json +if ([string]::IsNullOrEmpty($signResponse.signature)) { + Write-PipelineTelemetryError -Category 'Build' -Message "Key Vault returned an empty signature for key '$KeyName' in vault '$KeyVaultName'." + exit 1 +} +$signatureUrl = $signResponse.signature.TrimEnd('=').Replace('+', '-').Replace('/', '_') +$jwt = "$signingInput.$signatureUrl" + +$headers = @{ + Authorization = "Bearer $jwt" + 'X-GitHub-Api-Version' = '2022-11-28' + Accept = 'application/vnd.github+json' + 'User-Agent' = 'dotnet-arcade-onelocbuild' +} + +Write-Host "Looking up installation for '$InstallationOwner'..." +try { + $installations = Invoke-RestMethod -Uri 'https://api.github.com/app/installations' -Headers $headers -Method Get +} +catch { + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to list GitHub App installations: $_. The signed JWT may be invalid or the App's Client ID ('$AppClientId') may be incorrect." + exit 1 +} +$installation = $installations | Where-Object { $_.account.login -eq $InstallationOwner } | Select-Object -First 1 +if (-not $installation) { + $found = ($installations | ForEach-Object { $_.account.login }) -join ', ' + Write-PipelineTelemetryError -Category 'Build' -Message "No installation found for '$InstallationOwner'. App is installed on: $found" + exit 1 +} + +try { + $tokenResponse = Invoke-RestMethod ` + -Uri "https://api.github.com/app/installations/$($installation.id)/access_tokens" ` + -Headers $headers ` + -Method Post ` + -ContentType 'application/json' +} +catch { + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to mint an installation access token for '$InstallationOwner' (installation $($installation.id)): $_" + exit 1 +} + +Write-Host "Got installation token for '$InstallationOwner' (expires $($tokenResponse.expires_at))." +if ($OutputVariableName) { + Write-Host "Setting pipeline variable '$OutputVariableName'." + Write-Host "##vso[task.setvariable variable=$OutputVariableName;issecret=true]$($tokenResponse.token)" +} +else { + Write-Host $tokenResponse.token -ForegroundColor Green +} diff --git a/eng/common/core-templates/job/onelocbuild.yml b/eng/common/core-templates/job/onelocbuild.yml index 2816d2905a0..338dfb8ff68 100644 --- a/eng/common/core-templates/job/onelocbuild.yml +++ b/eng/common/core-templates/job/onelocbuild.yml @@ -14,6 +14,15 @@ parameters: # exist, and any pipeline that sets this to '' fall back to PAT-based auth via the CeapexPat parameter. CeapexServiceConnection: 'dnceng-onelocbuild-ceapex' + # GitHub App authentication for the OneLoc check-in PR (dnceng/internal only). + # The infrastructure identifiers are centralized here so consumers only need to opt in. + # DevDiv requires its own project-scoped service connection before this path can be enabled there. + UseGitHubAppAuthentication: false + GitHubAppServiceConnection: 'dnceng-oneloc-githubapp' + GitHubAppClientId: 'Iv23lijBU8x3gc9lDOc9' + GitHubAppKeyVaultName: 'EngKeyVault' + GitHubAppKeyName: 'oneloc-localization-app-key' + SourcesDirectory: $(System.DefaultWorkingDirectory) CreatePr: true AutoCompletePr: false @@ -89,6 +98,19 @@ jobs: outputVariableName: 'CeapexEntraToken' condition: ${{ parameters.condition }} + # Mint a short-lived GitHub App installation token for the loc check-in PR (dnceng/internal only). + # All other projects fall back to PAT-based auth, since the app service connection is scoped to dnceng/internal. + - ${{ if and(eq(parameters.RepoType, 'gitHub'), eq(parameters.UseGitHubAppAuthentication, true), eq(variables['System.TeamProject'], 'internal')) }}: + - template: /eng/common/templates/steps/get-github-app-token.yml + parameters: + azureSubscription: ${{ parameters.GitHubAppServiceConnection }} + keyVaultName: ${{ parameters.GitHubAppKeyVaultName }} + keyName: ${{ parameters.GitHubAppKeyName }} + appClientId: ${{ parameters.GitHubAppClientId }} + installationOwner: ${{ parameters.GitHubOrg }} + outputVariableName: 'GitHubAppInstallationToken' + condition: ${{ parameters.condition }} + - task: OneLocBuild@2 displayName: OneLocBuild env: @@ -110,7 +132,10 @@ jobs: patVariable: ${{ parameters.CeapexPat }} ${{ if eq(parameters.RepoType, 'gitHub') }}: repoType: ${{ parameters.RepoType }} - gitHubPatVariable: "${{ parameters.GithubPat }}" + ${{ if and(eq(parameters.UseGitHubAppAuthentication, true), eq(variables['System.TeamProject'], 'internal')) }}: + gitHubPatVariable: "$(GitHubAppInstallationToken)" + ${{ if or(eq(parameters.UseGitHubAppAuthentication, false), ne(variables['System.TeamProject'], 'internal')) }}: + gitHubPatVariable: "${{ parameters.GithubPat }}" ${{ if ne(parameters.MirrorRepo, '') }}: isMirrorRepoSelected: true gitHubOrganization: ${{ parameters.GitHubOrg }} diff --git a/eng/common/core-templates/steps/get-github-app-token.yml b/eng/common/core-templates/steps/get-github-app-token.yml new file mode 100644 index 00000000000..6d42a48d3c3 --- /dev/null +++ b/eng/common/core-templates/steps/get-github-app-token.yml @@ -0,0 +1,79 @@ +# Mints a short-lived GitHub App installation access token by signing a JWT +# with a private key stored in Azure Key Vault (RSA, RS256). The JWT is +# exchanged with the GitHub API for a token scoped to a single installation. +# +# Requirements (per GitHub App you want to authenticate as): +# - A GitHub App with its private key uploaded into Key Vault as an RSA key +# (PEM converted to a key, NOT stored as a secret). +# - The Azure service connection passed via `azureSubscription` must be +# granted the `Key Vault Crypto User` role (or at minimum `Sign` action) +# on that key. +# - The App must be installed on the target organization/account +# (`installationOwner`) with the permissions/repositories you need. +# +# Output: a secret pipeline variable named ${{ parameters.outputVariableName }} +# containing the installation access token. Token lifetime is ~1 hour and is +# automatically scrubbed from logs. Installation tokens are exempt from the +# enterprise classic-PAT lifetime policy. + +parameters: +# Azure DevOps service connection (federated) that can call +# `az keyvault key sign` on the App's signing key. +- name: azureSubscription + type: string + +# Name of the Key Vault that holds the GitHub App's RSA signing key. +- name: keyVaultName + type: string + +# Name of the RSA key inside the Key Vault (the App's private key). +- name: keyName + type: string + +# The GitHub App's Client ID (the value to put in the `iss` JWT claim). +# Prefer this over the numeric App ID; GitHub accepts either, but Client ID +# is the documented form going forward. +- name: appClientId + type: string + +# Login of the organization or user account whose installation we should +# mint the token for (e.g. `dotnet`, `microsoft`). +- name: installationOwner + type: string + +# Name of the pipeline variable that will receive the installation token. +- name: outputVariableName + type: string + +- name: is1ESPipeline + type: boolean + +- name: stepName + type: string + default: getGitHubAppInstallationToken + +- name: condition + type: string + default: '' + +- name: displayName + type: string + default: Get GitHub App installation token + +steps: +- task: AzureCLI@2 + displayName: ${{ parameters.displayName }} + name: ${{ parameters.stepName }} + ${{ if ne(parameters.condition, '') }}: + condition: ${{ parameters.condition }} + inputs: + azureSubscription: ${{ parameters.azureSubscription }} + scriptType: pscore + scriptLocation: inlineScript + inlineScript: | + & "$(System.DefaultWorkingDirectory)/eng/common/Get-GitHubAppToken.ps1" ` + -KeyVaultName '${{ parameters.keyVaultName }}' ` + -KeyName '${{ parameters.keyName }}' ` + -AppClientId '${{ parameters.appClientId }}' ` + -InstallationOwner '${{ parameters.installationOwner }}' ` + -OutputVariableName '${{ parameters.outputVariableName }}' diff --git a/eng/common/templates-official/steps/get-github-app-token.yml b/eng/common/templates-official/steps/get-github-app-token.yml new file mode 100644 index 00000000000..c89f3641a4d --- /dev/null +++ b/eng/common/templates-official/steps/get-github-app-token.yml @@ -0,0 +1,7 @@ +steps: +- template: /eng/common/core-templates/steps/get-github-app-token.yml + parameters: + is1ESPipeline: true + + ${{ each parameter in parameters }}: + ${{ parameter.key }}: ${{ parameter.value }} diff --git a/eng/common/templates/steps/get-github-app-token.yml b/eng/common/templates/steps/get-github-app-token.yml new file mode 100644 index 00000000000..79e182c6416 --- /dev/null +++ b/eng/common/templates/steps/get-github-app-token.yml @@ -0,0 +1,7 @@ +steps: +- template: /eng/common/core-templates/steps/get-github-app-token.yml + parameters: + is1ESPipeline: false + + ${{ each parameter in parameters }}: + ${{ parameter.key }}: ${{ parameter.value }}