Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: verify provenance #309

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 49 additions & 5 deletions .github/workflows/goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ jobs:
uses: actions/checkout@24cb9080177205b6e8c946b17badbe402adc938f # v3.4.0
with:
fetch-depth: 0
- name: Run Tests
uses: ./.github/workflows/test.yaml
- name: Run Lints
uses: ./.github/workflows/lint.yaml
# - name: Run Tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we bring these back? This is an important part of the release process.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reason why I did this is that workflow didn't worked as it didn't find these action files under .github/actions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah my bad, fixing this in #311.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@another-rex I think the problem still continues. I also do not think that this is the way of referencing different workflows within the workflow.

Screen Shot 2023-03-25 at 3 29 07 PM

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I misunderstood how the uses line works. I updated and tested it in #315, which seems to work in my fork.

# uses: ./.github/workflows/test.yaml
# - name: Run Lints
# uses: ./.github/workflows/lint.yaml
- uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # v2
- uses: docker/setup-buildx-action@4b4e9c3e2d4531116a6f8ba8e71fc6e2cb6e6c8c # v2
- name: Set up Go
Expand All @@ -49,9 +49,11 @@ jobs:
args: release --rm-dist
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OWNER: ${{ github.repository_owner }}
- name: Generate subject
id: hash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ARTIFACTS: "${{ steps.run-goreleaser.outputs.artifacts }}"
run: |
set -euo pipefail
Expand All @@ -63,7 +65,49 @@ jobs:
actions: read # To read the workflow path.
id-token: write # To sign the provenance.
contents: write # To add assets to a release.
uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v1.5.0
uses: developer-guy/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v1.5.0
with:
base64-subjects: "${{ needs.goreleaser.outputs.hashes }}"
upload-assets: true # upload to a new release
draft-release: true
verification:
needs: [goreleaser, provenance]
runs-on: ubuntu-latest
permissions:
contents: write # To add assets to a release.
Copy link

@laurentsimon laurentsimon Mar 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please file a bug on https://github.com/actions/runner. Requiring write permission is really odd

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me try it again, I changed back it to "read".

Copy link
Author

@developer-guy developer-guy Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

@developer-guy developer-guy Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this script is just working fine with write permission: https://github.com/developer-guy/osv-scanner/actions/runs/4519410595/jobs/7959907135

set -euo pipefail
  TAG=${GITHUB_REF#refs/tags/}
  echo "Downloading assets for tag $TAG"
  for i in $(gh api "repos/$GITHUB_REPOSITORY/releases" | jq -r '.[] | select(.tag_name=='\"$TAG\"').assets[].id')
  do
    echo "Downloading asset $i"
    curl -L -o $(gh api "repos/$GITHUB_REPOSITORY/releases/assets/$i" | jq -r '.name') -H "Authorization: token $GH_TOKEN" -H "Accept: application/octet-stream" $(gh api "repos/$GITHUB_REPOSITORY/releases/assets/$i" | jq -r '.url')
  done
  echo "Downloading provenance $PROVENANCE"
  gh -R $GITHUB_REPOSITORY release download --clobber $GITHUB_REF_NAME -p $PROVENANCE

steps:
- name: Install SLSA verifier
uses: slsa-framework/slsa-verifier/actions/installer@v2.1.0
- name: Download assets
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PROVENANCE: "${{ needs.provenance.outputs.provenance-name }}"
run: |
set -euo pipefail
TAG=${GITHUB_REF#refs/tags/}
Comment on lines +86 to +87
Copy link
Collaborator

@oliverchang oliverchang Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a fair bit of bash here to maintain :)

@laurentsimon @ianlewis is this recommended practice? If so, would it be possible at all to generalize this in a more re-usable way "upstream" somewhere as part of a slsa-framework repo?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it might be a good idea to have an action for verifying artifacts on a GitHub release. i.e. Given a release download all the artifact and verify them.

This would only work for a situation like osv-scanner where you provide binary artifacts along with the provenance (as opposed to some other language package) but I think it could still be useful.

We would also need to be careful to provide a way to provide the list of expected artifacts. We would need to catch the situation where an artifact was excluded from the provenance even though it was expected to be there (in this case goreleaser.outputs.hashes)

/cc @laurentsimon

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can work on this 🤩

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a wonderful idea we should discuss this topic

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianlewis @laurentsimon thoughts on this going into some slsa-framework repo instead (either a new one or existing?)

I would very much prefer that approach as it makes this work a lot more useful and we (OSV-Scanner maintainers) have less things we're not as familiar with to maintain :)

echo "Downloading assets for tag $TAG"
for i in $(gh api "repos/$GITHUB_REPOSITORY/releases" | jq -r '.[] | select(.tag_name=='\"$TAG\"').assets[].id')
do
echo "Downloading asset $i"
curl -L -o $(gh api "repos/$GITHUB_REPOSITORY/releases/assets/$i" | jq -r '.name') -H "Authorization: token $GH_TOKEN" -H "Accept: application/octet-stream" $(gh api "repos/$GITHUB_REPOSITORY/releases/assets/$i" | jq -r '.url')
done
Comment on lines +89 to +93

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you can't just do gh release download "$TAG" ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the same reason I explained above: #309 (comment)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it worked @ianlewis with right permisssions.

echo "Downloading provenance $PROVENANCE"
gh -R $GITHUB_REPOSITORY release download --clobber $GITHUB_REF_NAME -p $PROVENANCE
Comment on lines +94 to +95

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm understanding this code properly, the provenance would be already downloaded as it's also a release asset.

Copy link

@laurentsimon laurentsimon Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the code from https://github.com/google/go-containerregistry/blob/main/.github/workflows/release.yml#L48-L76 should work just fine, no?
We can make the attestation name dynamic instead of hard-coding it

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is the PR for that: google/go-containerregistry#1611

gh release download didn't work with draft releases as osv-scanner marks its release as draft and I looked for a solution for gh release download to enable downloading assets from draft release but I couldn't find anything useful, then I turned the solution back to the GitHub API calls via gh api. But let me try again to ensure that I didn't work.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it worked btw with the right permissions, so to download releases it requires you to put write permissions for content. Great!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's interesting. Maybe file an issue on GitHub.. sounds like a bug... they should have a REST flag to allow for draft release to be fetched, without the need for a write token.

- name: Verify assets
env:
CHECKSUMS: ${{ needs.goreleaser.outputs.hashes }}
PROVENANCE: "${{ needs.provenance.outputs.provenance-name }}"
run: |
set -euo pipefail
checksums=$(echo "$CHECKSUMS" | base64 -d)
while read -r line; do
fn=$(echo $line | cut -d ' ' -f2)
echo "Verifying $fn"
slsa-verifier verify-artifact "$fn" \
--provenance-path "$PROVENANCE" \
--source-uri "github.com/$GITHUB_REPOSITORY" \
--source-tag "$GITHUB_REF_NAME" \
--builder-id "https://github.com/developer-guy/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v1.5.0"
done <<<"$checksums"


16 changes: 8 additions & 8 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ builds:
dockers:
# Arch: amd64
- image_templates:
- 'ghcr.io/google/osv-scanner:{{ .Tag }}-amd64'
- 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}-amd64'
dockerfile: goreleaser.dockerfile
use: buildx
build_flag_templates:
Expand All @@ -49,7 +49,7 @@ dockers:
- "--platform=linux/amd64"
# Arch: arm64
- image_templates:
- 'ghcr.io/google/osv-scanner:{{ .Tag }}-arm64'
- 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}-arm64'
dockerfile: goreleaser.dockerfile
use: buildx
build_flag_templates:
Expand All @@ -67,14 +67,14 @@ dockers:
goarch: arm64

docker_manifests:
- name_template: 'ghcr.io/google/osv-scanner:{{ .Tag }}'
- name_template: 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}'
image_templates:
- 'ghcr.io/google/osv-scanner:{{ .Tag }}-amd64'
- 'ghcr.io/google/osv-scanner:{{ .Tag }}-arm64'
- name_template: 'ghcr.io/google/osv-scanner:latest'
- 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}-amd64'
- 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}-arm64'
- name_template: 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:latest'
image_templates:
- 'ghcr.io/google/osv-scanner:{{ .Tag }}-amd64'
- 'ghcr.io/google/osv-scanner:{{ .Tag }}-arm64'
- 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}-amd64'
- 'ghcr.io/{{ .Env.OWNER }}/osv-scanner:{{ .Tag }}-arm64'

archives:
- format: binary
Expand Down