ci: ship provenance as a release asset, add opt-in Codecov Test Analytics - #333
Merged
Conversation
actions/attest-build-provenance files the attestation in GitHub's
attestation store, where `gh attestation verify` finds it. Nothing about
the release itself says it carries provenance, so a consumer reading the
release — including OpenSSF Scorecard — cannot see it. Scorecard's
releasesHaveProvenance probe matches release assets on a `.intoto.jsonl`
suffix:
var provenanceExtensions = []string{".intoto.jsonl"}
so every release produced by this workflow scored as having none. The
warning on netresearch/ofelia's scorecard ("release artifact v0.28.1
does not have provenance") is that gap, not a missing attestation: the
v0.28.1 linux/amd64 binary does have one, retrievable by digest.
The bundle the action writes is a Sigstore bundle wrapping a DSSE
envelope. Renaming it to .intoto.jsonl would satisfy the probe while
misdescribing the format — a consumer expecting JSON Lines would get a
Sigstore bundle. The envelope is unwrapped instead, so the extension
describes what the file is: one DSSE envelope on one line, carrying the
same in-toto statement.
Verified against a real bundle (ofelia v0.28.1, linux/amd64): the
transformation yields a single line whose payloadType is
application/vnd.in-toto+json and whose payload decodes to an
in-toto Statement/v1 with predicateType https://slsa.dev/provenance/v1
for subject ofelia-linux-amd64. The step fails loudly when the bundle is
absent or carries no dsseEnvelope, rather than shipping an asset that
looks like provenance and contains "null".
The container job's bin/ preparation now drops these files alongside the
SBOMs. Both share the binary's name prefix, so without that the new file
would shadow the Dockerfile's `COPY bin/<app>-linux-*` — this change
would otherwise have broken every consumer's image build.
No change is needed in verify-release.yml: the new asset is signed by
the existing per-asset cosign loop and listed in checksums.txt by the
existing checksum step, which is what that workflow asserts.
Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>
The workflow uploads coverage.out and nothing else, so Codecov's Test Analytics page has no data for any consumer: no flaky-test detection, no per-test durations, no failure history. That page is what netresearch/ofelia's /tests/new link shows as unconfigured. With enable-test-results on, the unit tests run through gotestsum, which drives `go test -json` and writes a JUnit report alongside the coverage profile, and codecov/test-results-action uploads it. It is opt-in and off by default. The alternative — swapping the `go test` invocation for everyone — changes how every consumer's tests are executed to gain a reporting feature, and that is not a trade a shared workflow should make on their behalf. The existing step is left exactly as it is and the gotestsum path is a second, mutually exclusive step, so a consumer that has not opted in runs the same command it runs today. The upload is gated on !cancelled() rather than success: a run whose tests failed is precisely the one Test Analytics exists to record, and gating on success would upload only green runs and leave the flaky-test view blind to every failure. hashFiles() guards the case where the run died before the report was written. Verified locally that `gotestsum --junitfile junit.xml --format standard-verbose -- <the default test-flags> <packages>` produces both artifacts from one run: a valid JUnit report (36 tests) and the coverage profile the threshold step and the Codecov upload already consume. codecov/test-results-action is pinned to the commit v1.2.1 dereferences to (0fa95f0e), resolved through both the tag object and the commits API rather than transcribed. Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR updates shared GitHub Actions reusable workflows to (1) publish build provenance in a release-consumable format and (2) optionally upload unit test results to Codecov Test Analytics, addressing gaps surfaced across consumer repositories.
Changes:
- Emit DSSE provenance as a
.intoto.jsonlfile and include it with binary artifacts for release publishing. - Ensure container builds don’t accidentally include SBOM/provenance files that would interfere with Dockerfile
COPY bin/<app>-linux-*. - Add an opt-in gotestsum/JUnit flow and upload test results to Codecov Test Analytics.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.github/workflows/release-go-app.yml |
Emits DSSE provenance as a .intoto.jsonl asset and prevents SBOM/provenance files from shadowing container COPY patterns. |
.github/workflows/go-check.yml |
Adds opt-in gotestsum + JUnit generation and uploads test results to Codecov Test Analytics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Review findings on PR #333. codecov/test-results-action is deprecated. Its README opens with the notice and points at codecov/codecov-action with a report-type parameter, so the JUnit upload now reuses the action — and the SHA — that the coverage upload already pins. One fewer third-party action, and no new pin to maintain. The migration note spells the parameter `report-type`. The action does not define that name: at the pinned v7.0.0 SHA the input is `report_type`, with no hyphenated alias, so following the note verbatim would have passed an unknown input that Actions drops without a word and uploaded the JUnit report as coverage. Checked against action.yml at that exact SHA rather than at main. The gotestsum version is now a literal instead of a workflow input. SonarCloud is right that a variable version cannot be shown to be pinned, and a shared workflow should own the version of a tool it installs rather than leaving each consumer to choose one. The knob was speculative; the version is bumped here. Corrects the enable-test-results description as well: it claimed CODECOV_TOKEN is required, but this workflow declares the secret optional and Codecov also authenticates through its GitHub App. Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>
My previous commit assumed SonarCloud's "Go dependencies should be locked to verified versions" fired because the version came from a workflow input. It did not: the rule still fired on the literal `@v1.13.0`. This workflow already answers that rule one line further down, where go-licenses is installed at a full commit SHA with the tag in a comment. gotestsum now follows the same form. A tag can be repointed at another commit; a commit cannot, which is what "locked to a verified version" asks for. The commit was resolved through the refs API and cross-checked against the commits API, then verified by running the install: `go install gotest.tools/gotestsum@c4a0df2e...` produces a binary reporting "gotestsum version v1.13.0". Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two gaps surfaced by netresearch/ofelia's dashboards. Both are in shared workflows, so both affect every consumer — the second is opt-in for that reason.
1. Build provenance never reaches the release
actions/attest-build-provenancefiles the attestation in GitHub's attestation store, wheregh attestation verifyfinds it. Nothing about the release itself says it carries provenance, so a consumer reading the release cannot see it. OpenSSF Scorecard'sreleasesHaveProvenanceprobe matches release assets on a suffix:so every release this workflow produces scores as having none. Ofelia's scorecard warning — "release artifact v0.28.1 does not have provenance" — is that gap, not a missing attestation: the v0.28.1 linux/amd64 binary does have one, retrievable by digest.
Why not just rename the bundle. The file the action writes is a Sigstore bundle wrapping a DSSE envelope. Naming it
.intoto.jsonlwould satisfy the probe while misdescribing the format — a consumer expecting JSON Lines would get a Sigstore bundle. The envelope is unwrapped instead, so the extension describes what the file actually is: one DSSE envelope on one line, carrying the same in-toto statement.Verified against a real bundle (ofelia v0.28.1, linux/amd64):
The step fails loudly if the bundle is missing or has no
dsseEnvelope, rather than shipping an asset that looks like provenance and containsnull.One thing this change would have broken. The container job drops SBOMs from
bin/because they share the binary's name prefix and shadow the Dockerfile'sCOPY bin/<app>-linux-*. The new files have exactly the same problem, so that cleanup now covers them too — without it, every consumer's image build would have broken.No change needed in
verify-release.yml: the new asset is signed by the existing per-asset cosign loop and listed inchecksums.txtby the existing checksum step, which is what that workflow asserts.2. Codecov Test Analytics has no data
go-check.ymluploadscoverage.outand nothing else, so the Test Analytics page is empty for every consumer: no flaky-test detection, no per-test durations, no failure history.With
enable-test-results: truethe unit tests run through gotestsum — which drivesgo test -jsonand writes a JUnit report next to the coverage profile — andcodecov/test-results-actionuploads it.Off by default, and a separate step. Swapping the
go testinvocation for everyone changes how every consumer's tests execute in order to gain a reporting feature; that is not a trade a shared workflow should make on their behalf. The existing step is untouched and the gotestsum path is mutually exclusive with it, so a consumer that has not opted in runs exactly the command it runs today.The upload is gated on
!cancelled()rather than success — a run whose tests failed is precisely what Test Analytics is for, and gating on success would upload only green runs and leave the flaky view blind to every failure.Verified locally that the invocation produces both artifacts from one run: a valid JUnit report (36 tests) and the coverage profile the threshold step and the Codecov upload already consume.
Notes
codecov/test-results-actionis pinned to the commitv1.2.1dereferences to (0fa95f0e), resolved via both the tag object and the commits API rather than transcribed from the tag.actionlintreports no findings on either workflow; both parse as YAML.enable-test-results: true; a follow-up PR on netresearch/ofelia will do that once this lands.