Signed release binaries with SLSA signatures

[laurentsimon]

Hi,

I'm reaching out on behalf of the Open Source Security Foundation (openssf.org). We work on improving the security of critical open source projects like yours.

Together with GitHub, we designed a free, easy-to-use method of code signing. It will help your users verify that your release binaries were built from your repository’s workflow and not altered by anyone. It’s just a few lines of code, but it will make your project more secure against third-party tampering and attacks like Codecov and CTX.

This PR shows how to add this seamless code signing to your workflow. You don’t have to be a cryptography expert or learn complicated tools and verification is simple for your users.

You can read more on the SLSA blog. Please reach out if you have any questions!

[johanbrandhorst]

Hi, this looks amazing, thank you. Since this is a change to the release process, I'd like to get one of the other maintainers to have a look too before merging.

[achew22]

Hey Laurent, thanks for talking the time to do this! Exciting stuff.

A couple of questions before I would feel comfortable merging this in:

1. Who is managing the signing key?
2. Do we have a unique signing key for the project or are we signed by a larger common key?
3. Is the signing key soft hsm backed or hard hsm backed?
4. What does key revocation on our part look like? If we decide that a release is bad how do we publicly declare that?
5. How can we audit the access controls on the key that's being used to sign the binary?
6. What kind of public audit log is being produced for signing events?
7. How are you ensuring that signing operations aren't skipping the audit log step?
8. Do we have a way to back up the key used in our project in case this signing service disappears?

There is probably documentation that covers this, but I didn't see it when I quickly spidered through the links you provided. Apologies if I missed a link that elaborated on this

[laurentsimon]

Hey Laurent, thanks for talking the time to do this! Exciting stuff.

A couple of questions before I would feel comfortable merging this in:

Great questions. Let me answer inline.

1. Who is managing the signing key?

Keys used to sign are ephemeral and only valid for 10mn. It works like this:

1. The running workflow (uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v1.2.0 in the PR), generate an ECDSA private/public key pair.
2. Then it proves its identity to a remote CA called Fulcio using OIDC which is OAuth but also works for non-human. Note: Fulcio is the CA from Sigstore. This is the solution Npm and Maven are planning to adopt to sign packages.
3. Once Fulcio has the identity of the running workflow, it create an x509 cert attesting to the workflow's public key, and adds CN/AltSubjectName/other/validity information to identify the workflow.
4. The workflow signs the binary / metadata. It uploads the signature / certificate / hash to a transparency log (its name is "rekor"). This works similarly to the certificate transparency log used for the web PKI. We in fact use their code to run it (Trillian). Because the certificates are only valid for 10mn or so, the log also appends a signed timestamp. (During verification, the timestamps are verified with the log's signed timestamp and certificate validity period is also verified)

2. Do we have a unique signing key for the project or are we signed by a larger common key?

Keys are ephemeral, valid for ~10mn

3. Is the signing key soft hsm backed or hard hsm backed?

It is backed by Google KMS, which use hardware HSM.

4. What does key revocation on our part look like? If we decide that a release is bad how do we publicly declare that?

CRL-type feature is WIP. But given that keys a short-lived, the risk is fairly low in practice.

5. How can we audit the access controls on the key that's being used to sign the binary?

Let me cc folks that work on Sigstore. There is a "ceremony" for key rotation, at least 3 of the maintainers need to agree. It's also video-recorded, I think.

/cc @bobcallaway and @var-sdk who are maintainers

6. What kind of public audit log is being produced for signing events?

All events are stored in the transparency log I mentioned above (cert / hash / signature / timestamp). It's publicly available, and can be queried via REST API: sigstore.dev/swagger. We're working with external parties, including universities, to write monitors for the log. You can look at https://rekor.tlog.dev/?logIndex=0 for simple functionalities on the log.

7. How are you ensuring that signing operations aren't skipping the audit log step?

During verification, we verify that the signed entry is in the log.

8. Do we have a way to back up the key used in our project in case this signing service disappears?

There is no key associated with your project, because it's ephemeral. Let me know if I'm mis-understanding.

There is probably documentation that covers this, but I didn't see it when I quickly spidered through the links you provided. Apologies if I missed a link that elaborated on this

no problem. I did not cover what you asked in the PR, so totally understand.

Let me know if you have further questions. Thanks

[laurentsimon]

Let me know if you have more questions.

[johanbrandhorst]

I was waiting to hear from @bobcallaway or @var-sdk, thanks for your answers!

[laurentsimon]

re is WIP. But given that keys a short-lived, the risk is fairly low in practice.

5. How can we audit the access controls on the key that's being used to sign the binary?

Let me cc folks that work on Sigstore. There is a "ceremony" for key rotation, at least 3 of the maintainers need to agree. It's also video-recorded, I think.

I mis-understood the question here. The x509 cert is generated based on the OIDC token that GitHub provides to attest to the GitHub workflow (https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect). The CA (Fulcio) only generates the cert if the token verification passes. Because the signing key is ephemeral and never stored (and only available within the workflow, not to the CA), there is no "traditional ACL" on it.

Let me know if this answers the question. Thanks!

[laurentsimon]

@bobcallaway and @var-sdk did I miss anything important in the description?

@johanbrandhorst I threw a lot of information at you, I'm not sure all of it is "digestable". The specification may shed further light on the project.

Any other questions / concerns I can help clarify?

Fyi, we onboard some other popular projects recently, such as https://github.com/google/ko and https://github.com/GoogleContainerTools/jib

[johanbrandhorst]

I'm satisfied by your answers, thank you. I'd just appreciate hearing from someone working at sigstore as well.

[bobcallaway]

@johanbrandhorst Bob from sigstore (a co-founder and technical steering committee member for the project) here :)

@laurentsimon answered things correctly, happy to answer any additional questions you might have

[johanbrandhorst]

Hi Bob, thanks for confirming the answers from @laurentsimon, I'm satisfied that this will be a net benefit to our users.

[johanbrandhorst]

Thanks for your contribution!

[johanbrandhorst]

I'll cut a new release soon, still waiting for a few more PRs to get merged.

[laurentsimon]

Great, feel free to keep us posted for any improvement you may like. Thanks again!

[johanbrandhorst]

Looks like there was an error in the release job 😢: https://github.com/grpc-ecosystem/grpc-gateway/runs/7982650605?check_suite_focus=true.

expected at least one subject

[laurentsimon]

I'm able to reproduce the problem. This was working last time I tried. I verified that the artifacts output variable is empty (https://github.com/goreleaser/goreleaser-action/blob/master/action.yml#L31), but it should not. I have created this issue to ask them what's happening goreleaser/goreleaser-action#368. Either I'm missing something totally obvious, or a bug was introduced recently in their project.