Compilation of jenkins-contribution-* GO applications does not publish CodeCoverage report

[jmMeessen]

Service(s)

GitHub

Summary

The code coverage automation doesn't work anymore for the two GO tools (see #4017 ). It requires at least an API token so that the coverage data can be uploaded. See https://docs.codecov.com/docs/adding-the-codecov-token

It is a handy feature especially as it fails the build if new code is added via a PR and is not covered by 80% coverage. A good reminder not to forget automated tests.

This is not a blocking issue but the loss of a handy feature.

Reproduction steps

No response

[dduportal]

triaged: Related to #4017 , added to milestone, taken by @lemeurherve and/or @dduportal

[lemeurherve]

@jmMeessen according to the doc you linked, we should add an API token to the repository, obtained from the CodeCov account.

How can we access this CodeCov account?

[dduportal]

@jmMeessen according to the doc you linked, we should add an API token to the repository, obtained from the CodeCov account.

How can we access this CodeCov account?

I wonder if we should not create a codecov account for jenkins-infra WDYT?

[lemeurherve]

Signing up doesn't seem possible from an email: https://about.codecov.io/sign-up/

[lemeurherve]

Requested permissions when using GitHub to sign up:

WDTY of creating a GitHub account named like "jenkins-infra-service-account" (or something equivalent), with 2FA enabled via oathtool like for #4073, that could be used to subscribe to this type of service without any risk?

[dduportal]

Requested permissions when using GitHub to sign up:

WDTY of creating a GitHub account named like "jenkins-infra-service-account" (or something equivalent), with 2FA enabled via oathtool like for #4073, that could be used to subscribe to this type of service without any risk?

That is an excellent idea! Are you ok to take care of this account creation?

[lemeurherve]

Account created: https://github.com/jenkins-infra-service-account

Next steps:

• Enable 2FA
  • Enabled, 2FA private key and credentials stored in https://github.com/jenkins-infra/charts-secrets/commit/bee0af4db888b6a0b8a18d0e987636341a90e8e6 (private repo)
• Create a (free) CodeCov account
  
• Add CodeCov API token as CODECOV_TOKEN organization secret scoped to the two Go repositories

[lemeurherve]

CODECOV_TOKEN organization secret created and scoped to https://github.com/jenkins-infra/jenkins-contribution-aggregator and https://github.com/jenkins-infra/jenkins-contribution-extractor.

According to https://docs.codecov.com/docs/quick-start#prerequisites,

In order to get started, Codecov will need the following:

• Access to a repository with GitHub, Bitbucket, or GitLab.
• If you are a GitHub user you MUST install the Codecov GitHub app for your organization - Codecov can't function without it.

We need to install https://github.com/apps/codecov GitHub App to these two repositories.

WDYT @dduportal?

[dduportal]

CODECOV_TOKEN organization secret created and scoped to https://github.com/jenkins-infra/jenkins-contribution-aggregator and https://github.com/jenkins-infra/jenkins-contribution-extractor.

According to https://docs.codecov.com/docs/quick-start#prerequisites,

In order to get started, Codecov will need the following:

• Access to a repository with GitHub, Bitbucket, or GitLab.
• If you are a GitHub user you MUST install the Codecov GitHub app for your organization - Codecov can't function without it.

We need to install https://github.com/apps/codecov GitHub App to these two repositories.

WDYT @dduportal?

LGTM! Let's review the expected permission before performing the effective installation (e.g. you can start the process, check the requested permissions, report here and if it looks safe you'll be able to finish the installation)

[lemeurherve]

Requested permissions:

[dduportal]

Requested permissions:

LGTM!

[lemeurherve]

I installed the CodeCov GitHub App on these 2 repositories:

Working on this issue and on #4141 I noticed that previously defined repository secrets were imported, and were overriding organization secret in case of CODECOV_TOKEN:

I tried a GitHub Action run with the CODECOV_TOKEN repository secret removed in benefit of the CODECOV_TOKEN organization token set to the jenkins-infra-service-account token and scoped to these 2 repositories.
But it returned the following error cf https://github.com/jenkins-infra/jenkins-contribution-aggregator/actions/runs/9584808001/job/26440342954:

Commit creating failed: {"detail":"Not valid tokenless upload"}

As jenkins-infra-service-account is only a regular member of @jenkins-infra organization and not an admin, we can't generate a CodeCov API token covering jenkins-infra org.
Thus I tried setting a new CODECOV_TOKEN repository secret to the API token of a repository configured in CodeCov interface (still logged as jenkins-infra-service-account):

CodeCov:

GitHub:

And that worked, cf the "Upload coverage report to CodeCov" step of https://github.com/jenkins-infra/jenkins-contribution-aggregator/actions/runs/9584808001/job/26441218787

I then applied the same for the other repository:

• Configuring the repo in CodeCov interface
• Retrieving an API repo token from it
• Updated the existing CODECOV_TOKEN repository secret that was imported during the repo transfer

I reran a "Test" job to confirm CodeCov correct upload from this second repository: https://github.com/jenkins-infra/jenkins-contribution-extractor/actions/runs/9580302254/job/26441497327job/26441422043

And finally I removed the unused CODECOV_TOKEN organization secret that I created when I started working on this.

The CodeCov badges have been updated on both READMEs, from:

To:

Note: the free CodeCov account has only one personal seat but has up to 5 seats available for the jenkins-infra organization if needed (I unticked the "auto-activate member" setting which was enabled by default):

[lemeurherve]

Confirmed working for jenkins-contribution-aggregator, CodeCov report uploaded on jenkins-infra/jenkins-contribution-aggregator#40 merge:
https://github.com/jenkins-infra/jenkins-contribution-aggregator/actions/runs/9597358905/job/26466246588

info - 2024-06-20 12:24:35,439 -- Process Upload complete

[jmMeessen]

I confirm too that the CodeCoverage upload appears to work. The real test of the feature will be when opening a PR with actual code changes. I am thus closing this issue as completed.

Thank you @lemeurherve to have thoroughly investigated this and found the adequate solution. 👍