Skip to content

Fix Install-GitHubApp 404 in Organizations tests — missing enterprise_organization_installations permission #596

@MariusStorhaug

Description

The Organizations integration test creates a new enterprise organization under the msx enterprise and then attempts to install the GitHub App on it. The Install-GitHubApp call returns a 404, which cascades into failures for all subsequent tests that depend on the app installation context (connecting to the org, updating via org context, removing via org context).

Observed in run 25260449736, auth case "GitHub App from an Enterprise using JWT + IAT on enterprise account":

Error: [-] Install-GitHubApp - Installs a GitHub App to an organization 2.87s
URI: https://api.github.com/enterprises/msx/apps/organizations/Organizations-Linux-25260449736/installations
Status: 404 (Not Found)
x-accepted-github-permissions: enterprise_organization_installations=write

Cascading failures

The 404 causes four consecutive test failures:

  1. Install-GitHubApp — 404 from the enterprise endpoint
  2. Connect-GitHubApp$orgContext is null because the app was never installed
  3. Update-GitHubOrganization (org installation) — cannot connect because there is no org installation
  4. Remove-GitHubOrganization (org installation) — same cascading failure

Request

Investigation results

No organization name mismatch. Both the creation and installation calls use the same $orgName variable (Organizations-{OS}-{RunID}). The org is created via New-GitHubOrganization -Enterprise 'msx' -Name $orgName and installed via Install-GitHubApp -Enterprise $owner -Organization $orgName where $owner = 'msx'. The naming is consistent throughout the test.

Missing permission identified. The response header x-accepted-github-permissions: enterprise_organization_installations=write indicates the endpoint requires the enterprise_organization_installations permission. The enterprise installation access token (IAT) context shows its permissions as {enterprise_custom_properties, enterprise_ai_controls, enterprise_copilot_metrics, enterprise_credentials…} — the truncated list does not appear to include enterprise_organization_installations. GitHub returns 404 (rather than 403) when the token lacks the required permission, which is standard behavior to avoid revealing resource existence.

Alternatively, propagation delay. A secondary possibility is that the freshly-created enterprise organization has not yet propagated to the enterprise apps endpoint. If the permission is confirmed present, a small retry delay between org creation and app installation should be tested.

Acceptance criteria

  • The Install-GitHubApp test passes for the enterprise auth case
  • All downstream tests (Connect-GitHubApp, Update-GitHubOrganization, Remove-GitHubOrganization via org installation) pass as a result
  • The enterprise app's required permissions are documented in tests.instructions.md

Technical decisions

Root cause — permission configuration: The psmodule-enterprise-app GitHub App (ClientID: Iv23lieHcDQDwVV3alK1, homed in the msx enterprise) needs the enterprise_organization_installations permission set to write. This is configured in the app's settings on GitHub, not in code.

Verification approach: List the full permissions of the enterprise IAT by expanding the context output (the truncation in the log hides whether the permission is present). If the permission is absent, add it to the app settings. If present, investigate propagation delay and add a retry.

Test ordering is correct: The test intentionally runs Update-GitHubOrganization and Remove-GitHubOrganization with Should -Throw before Install-GitHubApp. These expect to fail because the enterprise IAT lacks organization_administration=write on the newly created org (by design — the point is to show that org-level operations require an org-level installation). The Install-GitHubApp test then installs the app to get an org-level IAT for the remaining tests.

No code change required (if permission is the cause): The fix is to update the GitHub App configuration on the msx enterprise to include enterprise_organization_installations: write.


Implementation plan

Investigation

  • Verify whether enterprise_organization_installations is present in the enterprise app permissions by checking the app settings at https://github.com/enterprises/msx/settings/apps or by querying the API

Fix

  • If missing: Add enterprise_organization_installations: write permission to the psmodule-enterprise-app in the msx enterprise settings
  • If present: Add a retry/delay between New-GitHubOrganization and Install-GitHubApp in Organizations.Tests.ps1 to handle propagation delay
  • Re-run the enterprise auth case in Organizations.Tests.ps1 to confirm all 4 cascading tests pass

Documentation

  • Add the required enterprise app permissions to tests.instructions.md under the APP_ENT section
  • Document that enterprise_organization_installations: write is needed for the Organizations test enterprise flow

Metadata

Metadata

Labels

bugSomething isn't workingpatch

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions