Support using legitify action with GITHUB_TOKEN

[noamd-legit]

TL;DR

Currently, it is not possible to use the action with the automatically generated GITHUB_TOKEN because it has no permission to execute the following API:

https://api.github.com/user/orgs

It should be possible to use it when executing the action with the analyze_self option

Detailed design

No response

Additional information

No response

[mackowski]

Hey @noamd-legit is there a plan to implement this? I belive this is also a blocker to use legitify in a GitHub App.

[molson504x]

@noamd-legit +1 for this - this is a blocker from a client being able to use this product because they aren't comfortable with using a PAT token in this instance.

[molson504x]

@noamd-legit - I did a bunch of digging into Legitify's source code, and I think I see why this isn't an "easy fix" scenario, and instead would require a pretty large refactor. And despite what we originally thought, it's NOT because they have to call out to the /user/orgs API endpoint. This is going to be a long post, but tl;dr it's way more than just one API call that has a dependency, and this is not an easy fix.

That all being said, I can't help but feel like this effort still needs to be done since GH PAT tokens are tied to a specific user, and in this case the user also has to have quite a high level of access to resources. This isn't ideal for a wide variety of reasons, a big one in my opinion being that if the user leaves the organization then the organization will lose its abilities to do Legitify scans (not to mention the "what-if" of a PAT token of a user with this level of access becoming compromised).

Now... onto the good stuff...

Part of the initialization logic has a GraphQL query to get the viewerPermissions attribute of the repositories (see internal/clients/github/client.go). This attribute will always return null for a GitHub App per the GraphQL Object Docs. So great, I decided to just update that code locally to support looking for an empty string, and Legitify ran successfully... Sorta. The results felt a little incomplete, so I checked the permissions_log.json file and quickly found why it felt so incomplete:

{
  "missing_permissions": {},
  "skipped_policies": {
    "actions_can_approve_pull_requests": {
      "legitify-test": "Missing permission: admin:org"
    },
    "code_review_by_two_members_not_required": {
      "legitify-test": "Missing permission: repo"
    },
    "code_review_not_limited_to_code_owners": {
      "legitify-test": "Missing permission: repo"
    },

// --EDITED OUT A BUNCH IN THE MIDDLE TO SAVE SPACE SINCE THERE WAS A LOT--

    "users_allowed_to_bypass_ruleset": {
      "legitify-test": "Missing permission: repo"
    },
    "vulnerability_alerts_not_enabled": {
      "legitify-test": "Missing permission: repo"
    }
  }
}

That's a lot of skipped rules. I dug deeper into the rule analysis logic and stumbled into the skipper logic (internal/analyzers/skippers/skippers.go) and saw it is reading the token scopes. Hmm, how's it collecting the token scopes?

More digging led me to the collectTokenScopes function in the GitHub Client logic (internal/clients/github/client.go). Tl;dr what this is doing is issuing an empty post call to the GraphQL URL for GitHub, and looking for the X-OAuth-Scopes header to come back. It then parses that header which sets the scopes in the context to have the appropriate TokenScopes set to true or false.

With a classic PAT token, this header is there and looks something like this:

x-oauth-scopes: admin:org, admin:org_hook, read:repo_hook, repo

With a GH App (or a fine-grained PAT token) those OAuth scopes are absent. This means that all of the scopes get to set to false (it actually just doesn't update the initialScopes function output).

Also, it looks like OAuth Scopes are also used to define the required scopes for each rule (such as this one or this one or this one). The skipper is reading the policy metadata and looking for the requiredScopes, and then comparing it against the OAuth scopes it got back from the X-OAuth-Scopes header earlier (by way of reading the TokenScopes stored in the client Context object) to ensure that the required scopes are present on the token BEFORE attempting to do any logical validation to determine whether a rule is going to pass or fail. If the scope(s) aren't present, it'll skip the rule.

And that's just looking at the GitHub logic, I didn't even touch the GitLab logic (though I'd be willing to bet it's similar). AND since some of this logic is shared between GitHub and GitLab, that would make it an even larger refactor, which is riskier to the product.

[noamd-legit]

Thank you for taking the time to write this detailed report @molson504x!
This is indeed the reason we don't support fine-grained and GitHub App Tokens currently.

The main problem is that when using GitHub API in many cases you can't know whether an API call failed to due insufficient permissions or because the resource doesn't exists. That caused a lot of problems in the beginning (because it showed many FAILED policies although the user just didn't have the required permissions) and why we decided the verify the scopes.

1.Since the GitHub API doesn't provide the scopes for other token types, I think the only option is to disable the distinction between FAILED and SKIPPED policies when running with a GitHub App Token.

2. For fine-grained tokens, we'll need to disable the distinction and scope Legitify runs only to repository misconfigurations.

WDYT?