Tokenless support in the new CLI (for public repos and forks)

[toddbaert]

Hello! I've recently updated a number of projects in our org to use the latest versions of the codecov action.

I see now that the codecov CLI is used to power this action, and that it requires a token. No problem, I went and got the token for the repositories in question, and all seemed to work... until we got a PR from a fork (something that happens quite frequently in our case). This results in a problem: secrets aren't available in PRs from forks (unless we use the pull_request_target trigger, which would allow malicious actors to potentially exfiltrate our secret(s)).

So my question is, as an open-source project, how am I supposed to use the new action? I want to run the coverage analysis on the new code (which is untrusted) but in order to do that, I need to run in a trusted environment to get access to the secret. Am I missing something? 🤔

[thomasrockhu-codecov]

@toddbaert this is a known and yet unsolved problem. We are investigating GitHub's OIDC to see if this will be a suitable replacement for token-less uploading (including forks)

[toddbaert]

@thomasrockhu-codecov Thanks!

For now we'll stick to v3! Happy to help you folks test v4 if you have a proposed solution for forks at some point.

Should I close this issue?

[thomasrockhu-codecov]

@toddbaert let's leave it open for now so that anyone watching will get an update if OIDC is the way to go for forks.

For everyone else, you can add the CODECOV_TOKEN from the Codecov repo settings page to your repo secrets. Then, update your GitHub Actions workflows:

...
uses: codecov/codecov-action@v4
env:
  CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
...

[toddbaert]

@toddbaert let's leave it open for now so that anyone watching will get an update if OIDC is the way to go for forks.

For everyone else, you can add the CODECOV_TOKEN from the Codecov repo settings page to your repo secrets. Then, update your GitHub Actions workflows:

...
uses: codecov/codecov-action@v4
env:
  CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
...

Yep, looks like there's some linked PRs/issues, so probably good to keep it open until there's a solution for forks. Thanks again!

[dolfinus]

you can add the CODECOV_TOKEN from the Codecov repo settings page to your repo secrets

But this still requires changing on:pull_request to on:pull_request_target to access secrets of original repo from a fork.

[JoshuaKGoldberg]

Given how many public repositories use the action in a tokenless way, I'm surprised at how sudden this rollout has felt. I would have thought a few proactive actions would have been done first, if possible:

• Letting the community know our token-less uploading was causing service issues
• Adding the token as a strongly recommended -but not required- option first
• Adding Codecov rate limiting so particularly active repositories don't hog the requests
• Changing the action to fail the build if it fails on token-less upload (instead of just warn and then pass)

Was this change communicated out to people ahead of time? The only reason I learned of this is that my repositories were no longer updating Codecov numbers (despite passing codecov-action builds).

It's of course understandable if the Codecov team doesn't have the bandwidth to work on those changes first. But letting us know early on could have saved some community pain, I think.

[mwestphal]

Please note codecov v3 doesn't seem to work anymore.

[mwestphal]

So I managed to fix this on my own CI.

Somehow, the current behavior is completely undocumented but this is pretty simple.

• non-fork workflow should use a token
• forked workflow should be token-less

Actually, codecov@v4 action detect forked workflow and use tokenless automatically in the case, even if a token is specified, the codecov output will tell you:

    info - 2024-04-28 03:27:00,660 -- The PR is happening in a forked repo. Using tokenless upload.

So here is how it looks in my CI:

    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v4
      with:
        token: ${{secrets.codecov_token}}
        fail_ci_if_error: true
        verbose: true
        use_oidc: false

• token is empty in forked workflow but it doesnt matter, codecov action just ignore it anyway.
• fail_ci_if_error: added this to avoid the previous situation where codecov@v3 started failing silently
• verbose : Great for debugging
• use_oidc: I dont think this is needed right now, but for now I want to make sure codecov doesnt try to use this

hth

[rohan-at-sentry]

@mwestphal - thanks for writing in, this was indeed documented before at https://docs.codecov.com/docs/codecov-uploader#supporting-tokenless-uploads.

I am going to try to augment the docs with more detail, and also make this more prominent in the readme for the action

[mwestphal]

Indeed, thanks for pointing out the doc!

make this more prominent in the readme for the action

I think this is why I missed it. The doc you linked is massive. I should have searched there for "tokenless" but I did not.
In any case, improving the action doc to cover such a classic usecase is worth it imo.

[hugovk]

I've started adding tokens to organisations. Unfortunately, my projects tend to be:

• under my personal account
• the only one in an org, so it's tedious to click through to add secrets
• or in an repo or org I don't have admin access to and the admins are unavailable

Is there a way to add a token to a personal account so it can be used by all repos, similar to adding an an org-level?

What are the downsides of adding tokens in some shared plaintext file?

[rohan-at-sentry]

@hugovk

For the following use-cases

under my personal account
the only one in an org, so it's tedious to click through to add secrets

you can grab the global upload token and either

• set them as an environment secret OR
• specify it during upload time
  • If using CLI in CI, you have to pass the token during upload upload-process -t<token>

For

or in an repo or org I don't have admin access to and the admins are unavailable

see if you can access the settings page in the codecov UI for the repo (https://app.codecov.io/gh//settings

If yes, then you can grab the token and

• specify it during upload time
  • If using CLI in CI, you have to pass the token during upload upload-process -t<token>

Re

What are the downsides of adding tokens in some shared plaintext file?

If you are doing this to ensure uploads from forks are not impacted, this should largely already be in place see above and docs.

As for risks, sharing your token means others can upload phony coverage and blow away your data.

[hugovk]

Thanks, I'll try these out.

under my personal account
the only one in an org, so it's tedious to click through to add secrets

you can grab the global upload token and either

• set them as an environment secret OR

• specify it during upload time

  • If using CLI in CI, you have to pass the token during upload upload-process -t<token>

I use the GitHub Action.

Can an environment secret be set in my user account for all my repos, or does it need repeating for each repo at https://github.com/my_username/my_repo/settings/secrets/actions and https://github.com/my_username/my_repo/settings/secrets/dependabot?

[rohan-at-sentry]

@hugovk - from their docs, Github doesn't appear to support setting secrets at an account level. If you have the ability to set secrets at the org level, then you can set the Global Upload Token for all repos without individually setting it for each repo.

[jsoref]

@rohan-at-sentry: correct, it's a really horrible idea to imagine secrets at an account level because people tend to willy-nilly fork repositories into their account, which is just inviting repositories to be written such that when they're forked they steal user's secrets.

Instead anyone interested in doing things should create their own (single-account) org, and move/fork things from their account to their org. A nice benefit is that organizations also give you the ability to see current runners (you can't do that for actions running in repositories in your user account...).

[rohan-at-sentry]

@jsoref exactly!

[ljharb]

Forcing a move to an org is a pretty unacceptable ask for codecov usability, though - while it sucks that GitHub treats user-owned repos as second-class, user-owned repos need love too :-)

[rohan-at-sentry]

Forcing a move to an org is a pretty unacceptable ask for codecov usability

I agree, - to be clear I'm not advocating for user-owned repos to move to orgs (and I can see how my response could be interpreted that way).

[jsoref]

In theory, OIDC is the right way to go...

• it can do the right thing in an original repository or for commits in a fork of an original repository (secrets will do the wrong thing in forks, in that they'll be missing, and then you'll hit rate limits and things will fail)
• it's true that it won't handle PRs from across forks (for pull_request)
• if it actually worked and people were really careful and it was willing to allow it, it could work if people used pull_request_target, but getting that right would require two jobs, one for doing the testing and one for uploading the report (and the current code explicitly blocks this use case so that'd have to be removed...)
• but at least in my tests it codecov-actions' OIDC implementation doesn't appear to work and the error reporting from codecov's server isn't particularly helpful.

I'm not strongly advocating for using orgs, but I personally have moved to doing most of my work in them because the security benefits and additional concurrent runners / visibility I get from runners make them worthwhile.