Epic: Support GitHub Enterprise

[jldec]

Gitpod users would like to use their own GitHub Enterprise instance to work with Gitpod.

The experience of registering and connecting users to GHE should be at least on parity with private GitLab instances.

Support for registering a GHE instance and performing authentication is already in place.

Gaps (additional validation required)

• Registration of the GitHub App for Gitpod, for GHE instances.
• Context URL parsing
• Webhooks integration for prebuilds and projects
• Gitpod self-hosted create workspace parse .gitpod.yml error from Github Enterprise #8429

[gtsiolis]

See also #5297. Cc @jldec

[jankeromnes]

Logging my progress. Please let me know if I should do this somewhere else. 🙏

Current state

• We have a GitHub Enterprise installation for testing (thanks @metcalfc!)
• There are already docs for connecting GitHub Enterprise:
  • Seems to be written for Self-Hosted (e.g. "starting your Gitpod installation, you’ll get redirected to /first-steps")
  • However, it seems to work with gitpod.io
• I was able to connect our GitHub Enterprise instance with gitpod.io :

Problems

• However, the final step of adding the GitHub Enterprise integration is to Connect with it, and this currently fails:

Maybe it's because I've configured it badly, or there is a bug somewhere. Currently investigating this.

[jankeromnes]

This logged error seems relevant:

TokenError: The client_id and/or client_secret passed are incorrect.
    at GenericOAuth2Strategy.OAuth2Strategy.parseErrorResponse (/app/node_modules/passport-oauth2/lib/strategy.js:373:12)
    at GenericOAuth2Strategy.OAuth2Strategy._createOAuthError (/app/node_modules/passport-oauth2/lib/strategy.js:420:16)
    at /app/node_modules/passport-oauth2/lib/strategy.js:177:45
    at patchedCallback (/app/node_modules/@gitpod/server/dist/src/auth/generic-auth-provider.js:674:28)
    at /app/node_modules/oauth/lib/oauth2.js:209:7
    at passBackControl (/app/node_modules/oauth/lib/oauth2.js:134:9)
    at IncomingMessage.<anonymous> (/app/node_modules/oauth/lib/oauth2.js:157:7)
    at IncomingMessage.emit (node:events:402:35)
    at IncomingMessage.emit (node:domain:475:12)
    at endReadableNT (node:internal/streams/readable:1343:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Some other relevant snippets from around the error:

{
    "message": "(Auth-With-ghe.gitpod-self-hosted.com) Initialized.",
    "payload": {
        "defaultStrategyOptions": {
            "authorizationURL": "https://ghe.gitpod-self-hosted.com/login/oauth/authorize",
            "tokenURL": "https://ghe.gitpod-self-hosted.com/login/oaut/access_token",
            "clientID": "[REDACTED]",
            "clientSecret": " [REDACTED]",
            "callbackURL": "https://gitpod.io/auth/ghe.gitpod-self-hosted.com/callback",
            "scope": "user:email,read:user,public_repo,repo,read:org,workflow",
            "scopeSeparator": ",",
            "userAgent": "gitpod.io",
            "passReqToCallback": true
        }
    }
}

authFlow: {
    host: 'ghe.gitpod-self-hosted.com',
    returnTo: 'https://gitpod.io/complete-auth?message=success',
    overrideScopes: false
}

{
    path: '/auth/ghe.gitpod-self-hosted.com/callback?code=[REDACTED]'
}

{
    code: 'incorrect_client_credentials',
    uri: 'https://docs.github.com/enterprise/3.3/apps/managing-oauth-apps/troubleshooting-oauth-app-access-token-request-errors/#incorrect-client-credentials',
    status: 500
}

From what I can tell, the client_id and client_secret values look correct (except for a space in front of the secret? 🤔)

EDIT: It was the space in the secret 🤦 🤦 🤦

[jankeromnes]

Okay, connecting the app worked, let's try opening a repository.

Opening a private repository fails with:

Clicking on Grant Access just goes back to the same error again. New thing to debug! 🙌

EDIT:

• Granting public_repo and repo manually from the /integrations page works fine. Bug in the Grant Access flow?
• The next error is about parsing the .gitpod.yml -- since there is no .gitpod.yml in the repository, this is likely a bug in fetchConfig (I'm guessing it's trying to parse some HTML error page as YAML) Support GitHub Enterprise #8574

[AlexTugarev]

@jankeromnes and @jldec, what is the plan of registering of GitHub Apps?

It is a prerequisite for Projects support for GHE. Is it planned to extend the form on the /integrations page and use the DB to persist the private key?

[jankeromnes]

While GitHub Enterprise is now generally supported in both SaaS and Self-Hosted (#8574 🎉), I'll just briefly re-open this Epic for one more test:

It would be cool to experiment with a Gitpod Self-Hosted installation that points its GitHub App (singleton) to a GitHub Enterprise installation instead of github.com (for the particular case where you self-host Gitpod and only use your GitHub Enterprise installation without caring about github.com -- there it could potentially be helpful to set up the GitHub App in your GitHub Enterprise server in addition to the OAuth Integration).