[bug] Renovate upgrades to v1.3.0: no assets to download

[konstruktoid]

Describe the bug

Run ./__BUILDER_CHECKOUT_DIR__/.github/actions/generate-builder/generate-builder.sh
  ./__BUILDER_CHECKOUT_DIR__/.github/actions/generate-builder/generate-builder.sh
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    BUILDER_BINARY: slsa-generator-generic-linux-amd64
    BUILDER_DIR: ./__BUILDER_CHECKOUT_DIR__/internal/builders/generic
    BUILDER_REPOSITORY: slsa-framework/slsa-github-generator
    BUILDER_RELEASE_BINARY: slsa-generator-generic-linux-amd64
    VERIFIER_REPOSITORY: slsa-framework/slsa-verifier
    VERIFIER_RELEASE_BINARY: slsa-verifier-linux-amd64
    VERIFIER_RELEASE_BINARY_SHA256: b1d6c9bbce6274e253f0be33158cacd7fb894c5ebd643f14a911bfe55574f4c0
    VERIFIER_RELEASE: v1.3.2
    COMPILE_BUILDER: false
    BUILDER_REF: refs/tags/v1.3.0
    GH_TOKEN: ***
Fetching the builder with ref: refs/tags/v1.3.0
Builder version: v1.3.0
BUILDER_REPOSITORY: slsa-framework/slsa-github-generator
no assets to download
Error: Process completed with exit code 10.

https://github.com/konstruktoid/ansible-role-hardening/actions/runs/3512750596/jobs/5884806849#step:2:557
https://github.com/konstruktoid/ansible-role-docker-rootless/actions/runs/3514186184/jobs/5887905261#step:2:557
https://github.com/konstruktoid/hardening/actions/runs/3514179542/jobs/5887890423#step:2:557

Additional context

Renovate upgrades slsa-framework/slsa-github-generator action to v1.3.0

konstruktoid/ansible-role-hardening#208
konstruktoid/ansible-role-docker-rootless#90
konstruktoid/hardening#199

[laurentsimon]

Thanks. The v1.3.0 is marked as "pre-release", until we run all the pre-release tests. Can renovatebot be configured to not send PRs in this case? I know that dependabot does not, for example. We have this issue to also update the documentation #1219

[laurentsimon]

@rarkins Could you point us in the right direction?

[rarkins]

In future I recommend against using stable semver versions for pre-releases, as you may confuse humans and bots alike.

Renovate is upgrading tags here, and tags don't have prerelease designations - only releases do. If you are using either assets or release metadata (prerelease) then we'd need to switch to using GitHub releases only and no longer tags (even though GitHub Actions works off tags, as far as I'm aware). This means any tag you intend to be stable but which is missing a release would be ignored, as well as any release you designate as unstable.

[ianlewis]

@rarkins Thanks for the suggestions. Unfortunately we need to do pre-release tests which require exercising our verifier which requires us to use semver tags. We uncheck the pre-release checkbox after pre-release tests are done. That's really the only way for us to properly test the releases.

As a GHA CI/CD tool, we need to test after the tag is created and the reusable workflow is tied to this repository. So running tests in another repository risks us breaking production releases because of differences in repo settings etc. and we can't create separate non-semver tags for pre-releases because our SLSA verifier requires semver tags.

It feels like we're a strange edge case for this kind of thing...

[rarkins]

It seems like Renovate should switch to looking up releases instead of tags, in which case this pre-release flag is known and such PRs shouldn't be raised. I created a bug on Renovate and after confirmation from my co-maintainers we'll try to prioritize this change getting into production quickly: renovatebot/renovate#19033

[konstruktoid]

Thanks @rarkins, let see what happens

[HonkingGoose]

Heads-up: we closed the Renovate issue. It looks like the problem is not with Renovate: ¹

GitHub actions uses git refs, not GitHub releases. From the documentation:

The location and version of a reusable workflow file to run as a job. Use one of the following syntaxes:

• {owner}/{repo}/.github/workflows/{filename}@{ref} for reusable workflows in public repositories.
• ./.github/workflows/{filename} for reusable workflows in the same repository.

{ref} can be a SHA, a release tag, or a branch name.

GitHub releases are unrelated to GitHub Actions. GitHub releases are required to publish an action to the marketplace, but the marketplace is only for discoverability.

The issue is the slsa-github-generator expects there to be an equivalent GitHub release:

slsa-github-generator/.github/actions/generate-builder/builder-fetch.sh

Line 65 in 800ff15

gh release -R "$BUILDER_REPOSITORY" download "$BUILDER_TAG" -p "$BUILDER_RELEASE_BINARY*" || exit 10

Footnotes

1. https://github.com/renovatebot/renovate/issues/19033#issuecomment-1324159757 ↩

[rarkins]

To resolve this problem for SLSA we'll instead move this forward: renovatebot/renovate#19032

What this means is that we'll supplement github tags with github releases metadata whenever they correlate, which means we'll learn that 3.0.0 in this case is pre-release and therefore suppress it by default

[laurentsimon]

Thank you for the quick follow-up. Do you know how hard it is to implement and the timeline you envisage?

[rarkins]

I estimate 1-2 weeks

[konstruktoid]

This works as expected after updating, using renovate, slsa-framework/slsa-github-generator action to v1.4.0

[laurentsimon]

Thanks @konstruktoid
We have also changed how we do the release, and we use vX.Y.Z-rc to further mitigate this problem. I'll keep this issue opened until @rarkins has landed the new feature from renovate.

[ianlewis]

I think this should be solved now that renovatebot/renovate#19032 was finished and we now do RC releases to mitigate this issue.