Sign released ko binaries

[developer-guy]

Let's consider adding a signing step for ko binaries and container images with cosign? Or we should sign a checksum file as GoReleaser did ¹

cc: @Dentrax @imjasonh @jonjohnsonjr

Footnotes

1. https://carlosbecker.com/posts/goreleaser-cosign/ ↩

[imjasonh]

Duplicate of #357

[Dentrax]

I think what @developer-guy meant is that we can sign the checksums.txt to create a detached signature for the checksum: i.e. checksums.txt.sig, as GoReleaser did.

[imjasonh]

Thanks for clarifying, I think we should do this.

[developer-guy]

We can work on it if you want. We are the volunteers with @Dentrax. Can you please assign it to me, if you don't mind?

[imjasonh]

Is this something that goreleaser is interested in adding support for directly? Instead of having a cosign.key I'd probably just prefer to use GitHub's OIDC, which would be really smooth to integrate with goreleaser's GitHub Actions support.

In addition to signing the checksums.txt we should include verification instructions in release notes.

[developer-guy]

I think the main question is what will be signed, is the container image or the binary itself, right? Because the rest of it is just a method, we can use GoReleaser to do that, we can use GitHub's OIDC method for keyless signing, etc.

[imjasonh]

This project doesn't currently release an official container image (ironically), so I'd like to keep this focused on how best to sign the binaries goreleaser builds.

For that it sounds like we have a few options:

1. sign the released tar files (cosign sign-blob ko_0.9.3_Linux_x86_64.tar.gz)
2. sign the released binaries, inside the tar files (tar xvf ... && cosign sign-blob /tmp/ko)
3. sign the released checksums.txt, which ensures those checksums point to the right released artifacts -- this is what the blog post does.

There are probably other opitions. These three are all (AIUI) roughly the same in terms of the assurance they provide that a binary being downloaded hasn't been put there by some bad actor (unless that bad actor also controlled the release process that signed those things...).

Whatever we choose, the release notes should document steps to verify the signatures.

Longer term, I'd like to arrive at some best practice for signing released artifacts and have that baked into goreleaser, using GitHub OIDC, since I think that's the strongest assurance with the least amount of end-user effort.

[Dentrax]

Here is what we (@developer-guy) achieved so far:

0. Create a generate-key-pair usign cosign, and add password to the GitHub Secrets
1. Added sign and extraFiles stages in the goreleaser.yaml
2. Added cosign install stage in the release.yaml
3. Finally, released the version here

Here is the PoC result:

$ cosign verify-blob \
--key https://raw.githubusercontent.com/Dentrax/ko/feature/sign-checksum/cosign.pub \
--signature https://github.com/Dentrax/ko/releases/download/v0.9.3-signchecksum/checksums.txt.sig \
https://github.com/Dentrax/ko/releases/download/v0.9.3-signchecksum/checksums.txt

Verified OK

[imjasonh]

Nice!

Is there a way to use the keyless OIDC mode instead of generating a key and keeping it in a secret? It looks like signs doesn't let us set COSIGN_EXPERIMENTAL=1

edit: Hmm, cosign sign-blob supports the OIDC flow, but verify-blob doesn't seem to 🤔

[developer-guy]

Yes, if we use GoReleaser, we can't use this keyless OIDC mode; what if adding this support to GoReleaser? 🤩

[developer-guy]

In the meantime, this implement would work, but for later, I agree with you, using Keyless OIDC mode would be more beneficial.

[Dentrax]

Hey, @caarlos0! Should we create a proposal for this? ^ It would be cool!

[caarlos0]

yeah we should probably have a env field on signs 👍

feel free to open an issue, thankss

[developer-guy]

We found this example @imjasonh and want to try it out. Thanks to @shibumi 👏

in-toto/in-toto-golang#128

[developer-guy]

We have now the Keyless OIDC approach enabled with the latest commit. cosign v1.3.0 does not support verifying blob with the keyless approach but the code that enables it is already merged into main, so, once you install cosign from the main branch, now, you can be able to verify your signature like the following commands:

$ go install github.com/sigstore/cosign/cmd/cosign@main
$ httpie --download https://github.com/Dentrax/ko/releases/download/v0.9.3-signoidc/checksums.txt
$ httpie --download https://github.com/Dentrax/ko/releases/download/v0.9.3-signoidc/checksums.txt.sig
$ cosign verify-blob --signature checksums.txt.sig checksums.txt
Certificate is trusted by Fulcio Root CA
Email: []
Verified OK
tlog entry verified with uuid: "b30b5d3cd2a5f544c763aa91bb064d32861d5ccd1355179d939b9e0617c42d45" index: 841667

please see the GA pipeline result.

[shibumi]

It might make sense to wait for this feature in goreleaser: goreleaser/goreleaser#2659

With the newest changes in goreleaser + cosign it will be possible to directly release the public certificate to Github. I am really excited :D

[mattmoor]

🤔 Have all the pieces for this landed now? 🤩

[shibumi]

I think the cosign output-certificate support is missing (will be in v1.4.0), but besides this it should be possible to sign ko releases with cosign and goreleaser via workload identities.

[caarlos0]

GoReleaser also has a small bug related to this on v1.0.0. Already fixed on main, should land soon 🙏

[mattmoor]

This should be RFAL right?

[shibumi]

Would love to see this. My current workaround is this one here:

      - uses: imjasonh/setup-ko@v0.4
      - name: build and release image
        run: cosign sign --oidc-issuer=https://token.actions.githubusercontent.com $(ko publish ./ --tags  ${{github.ref_name}})
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          COSIGN_EXPERIMENTAL: 1

[mattmoor]

@shibumi I think you are after: #357

Also you shouldn't need --oidc-issuer

[shibumi]

@mattmoor can cosign autodetect the github issuer?

[shibumi]

@shibumi I think you are after: #357

ah yes, you are right. Too many tabs. Sorry for the noise.

[mattmoor]

can cosign autodetect the github issuer?

Yep: https://github.com/sigstore/cosign/blob/1feacab43fb01662ff372ab58e4186cf7da8ffcb/pkg/providers/github/github.go#L41

[developer-guy]

let's remove it then because we already tried that one in GoReleaser project, and everything worked fine 🤝

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.