CI: Add GitHub artifact attestations to package distribution #7427

matthewfeickert · 2024-05-23T17:17:16Z

Description

Add generation of GitHub artifact attestations to built sdist and wheels before upload to PyPI. c.f.:
- https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/
- https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds

This PR is related to scientific-python/summit-2024#9.

Checklist

A descriptive but concise pull request title
[N/A] Docstrings for all functions
[N/A] Unit tests
[N/A] A gallery example in ./doc/examples for new features
Contribution guide is followed

Release note

For maintainers and optionally contributors, please refer to the instructions on how to document this PR for the release notes.

Add generation of GitHub artifact attestations to built sdist and wheels before upload to PyPI.

.github/workflows/wheel_tests_and_release.yml

matthewfeickert · 2024-05-23T17:22:32Z

.github/workflows/wheel_tests_and_release.yml

+      - name: Verify the distribution
+        run: pipx run twine check --strict dist/*


Beyond this workflow, the only area twine is used is

scikit-image/tools/upload_wheels.sh

Lines 3 to 5 in 765e568

if [ "`which twine`" == "" ]; then

echo "twine not on path; need to pip install twine?"

exit 1

and tools/upload_wheels.sh seems to no longer be used anywhere in this repository

$ git grep "twine" .github/workflows/wheel_tests_and_release.yml: run: pipx run twine check --strict dist/* tools/upload_wheels.sh:if [ "`which twine`" == "" ]; then tools/upload_wheels.sh: echo "twine not on path; need to pip install twine?" $ git grep "upload_wheels"

so instead of installing twine into the global environment can just use the pre-installed instance of pipx on the runner to get it (in a virtual environment) for a distribution metadata check.

Should we not pin the version of twine, to avoid the threat of a hacked version of twine modifying uploaded wheel?

This version of twine doesn't do anything with the upload. All of that is handled through the pypa/gh-action-pypi-publish GitHub action below.

If you are also concerned with twine getting compromised and then modifying the wheel during the twine check command then you could pin to a trusted version, though I don't necessarily know if that's needed. If you were concerned you could also do a checksum before and after twine check ran to verify that it hasn't changed the distributions at all.

I'm probably being paranoid; we install a lot of tools that can be compromised. Somehow, in my head, that one seemed like a good target because it does uploading. Feel free to do the checksumming, but not necessary.

You're not wrong, so we could do this: #7427 (comment)

(Posting here as I messed up the view on the review tab)

matthewfeickert · 2024-05-23T17:23:00Z

.github/workflows/wheel_tests_and_release.yml

+      - name: Generate artifact attestation for sdist and wheels
+        uses: actions/attest-build-provenance@173725a1209d09b31f9d30a3890cf2757ebbff0d # v1.1.2
+        with:
+          subject-path: "dist/scikit_image-*"


c.f. https://github.com/actions/attest-build-provenance for more information.

Once this runs during a release the attestations will be uploaded to https://github.com/scikit-image/scikit-image/attestations and can be verified from a wheel or sdist artifact using the gh attestation verify CLI API. c.f. scikit-hep/pyhf#2473 for examples of that.

matthewfeickert · 2024-05-23T17:26:52Z

Given the maintenance history of this workflow (https://github.com/scikit-image/scikit-image/commits/main/.github/workflows/wheel_tests_and_release.yml) tagging @jarrodmillman for review.

stefanv · 2024-05-23T17:41:55Z

That first document is clear as mud, unless you know all the acronymns, so here is the tl;dr from it's summary:

attestations provide you with a paper trail, which cannot be forged, tying a given artifact to a given GitHub Actions workflow run.

stefanv

Thanks for this @matthewfeickert!

.github/workflows/wheel_tests_and_release.yml

stefanv · 2024-05-23T17:44:29Z

.github/workflows/wheel_tests_and_release.yml

+      - name: Verify the distribution
+        run: pipx run twine check --strict dist/*


Should we not pin the version of twine, to avoid the threat of a hacked version of twine modifying uploaded wheel?

* Add generation of GitHub artifact attestations to built sdist and wheels before upload to PyPI. c.f.: - https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/ - https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds

matthewfeickert · 2024-05-23T18:54:02Z

.github/workflows/wheel_tests_and_release.yml

+      - name: Verify the distribution
+        run: pipx run twine check --strict dist/*
+
+      # Ensure that a compromised twine couldn't have altered the distributions
+      # Required to resolved sdist and whl separately
+      - name: Verify sdist artifact attestation
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: gh attestation verify dist/scikit_image-*.tar.gz --repo ${{ github.repository }}
+
+      - name: Verify wheel artifact attestation
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: gh attestation verify dist/scikit_image-*.whl --repo ${{ github.repository }}


Instead of manually doing checksum validation, validate with twine after the attestations are created, and then use the attestations just created and published to validate that the distributions are unaltered before upload to PyPI.

YES, that is so cool 😎

* Add verification of artifact attestation before publishing to PyPI using the 'gh attestation verify' CLI API, added in v2.49.0. - c.f. https://github.com/cli/cli/releases/tag/v2.49.0 * If twine became compromised and altered the distributions _after_ they had an attestation created for them then the attestation check would fail.

matthewfeickert · 2024-05-23T19:08:35Z

pre-commit.ci was failing given a timeout, so repushed to retrigger.

stefanv · 2024-05-23T19:26:27Z

.github/workflows/wheel_tests_and_release.yml

@@ -22,6 +22,7 @@ jobs:
    permissions:
      contents: write # for softprops/action-gh-release to create GitHub release
      id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
+      attestations: write # for GitHub attestations


Oh, so they do require write access? 🙃

Yeah, they require the following permissions:

permissions: id-token: write attestations: write

c.f. https://github.com/actions/attest-build-provenance?tab=readme-ov-file#usage

stefanv · 2024-05-23T19:45:12Z

@lagru @jarrodmillman I think this is RTM.

lagru

@matthewfeickert, thanks for working on this! This seems useful and like an improvement. So I'm approving, but I'd still like to clarify a few things.

Even after reading the linked documentation, I'm still a bit unclear on how this compares to "just signing the artifact" (and some context info) with a key and uploading the signature somewhere to check against...

lagru · 2024-05-25T10:42:28Z

.github/workflows/wheel_tests_and_release.yml

@@ -48,6 +45,26 @@ jobs:
          python -m build --no-isolation --skip-dependency-check --sdist .
          ls -la ${{ github.workspace }}/dist

+      - name: Generate artifact attestation for sdist and wheels
+        uses: actions/attest-build-provenance@173725a1209d09b31f9d30a3890cf2757ebbff0d # v1.1.2


Would it make sense to create the attestation as close to artifact generation as possible, e.g. in the job call-workflow-build-wheels? Ideally, before anything else like the test suite is run on the created wheel?

@lagru Good question/point. I think that to some degree I would say yes, but if you want to only create attestations for releases and not just every run of the CI then you would either need to totally refactor the

jobs: call-workflow-build-wheels: uses: ./.github/workflows/wheels_recipe.yml

workflow to know about release situations (probably not worth it) or just do the slightly less perfect situation of this PR (signing the wheels just after download...the sdist is signed immediatley after build here at least).

Okay, I thing we can probably come up with a solution that uses inputs to enable artifacts in wheels_recipe only for releases. But let's do that in another PR (I'll open an issue) and merge this for now.

matthewfeickert · 2024-05-25T15:07:43Z

Even after reading the linked documentation, I'm still a bit unclear on how this compares to "just signing the artifact" (and some context info) with a key and uploading the signature somewhere to check against

@lagru Sorry, I don't think I fully understand what you're asking, as when I read this it seems like you're asking "Why don't I just become my own security provider?". I don't think that's what you're actually meaning though, so maybe you can clarify and I can leave this overview page for Sigstore (which is the underlying service that GitHub uses/owns): https://www.sigstore.dev/how-it-works

edit: To be fully transparent, I am not a security expert, and I can not give a comprehensive and short summary of how all the underlying infrastructure works in a single comment (you'll have to read the docs for that). However, if having a comprehensive executive summary would be useful I can go dig around on the internet and find one for you.

lagru · 2024-06-04T13:51:25Z

Thank you @matthewfeickert!

matthewfeickert commented May 23, 2024

View reviewed changes

stefanv approved these changes May 23, 2024

View reviewed changes

matthewfeickert force-pushed the ci/add-artifact-attestations branch from 20eb44f to 36e5753 Compare May 23, 2024 17:54

stefanv added the 🔧 type: Maintenance Refactoring and maintenance of internals label May 23, 2024

matthewfeickert force-pushed the ci/add-artifact-attestations branch from 4dbbab8 to 9ecc29b Compare May 23, 2024 18:52

matthewfeickert commented May 23, 2024

View reviewed changes

matthewfeickert requested a review from stefanv May 23, 2024 18:54

matthewfeickert force-pushed the ci/add-artifact-attestations branch from adf53d5 to b0b1d92 Compare May 23, 2024 19:08

stefanv approved these changes May 23, 2024

View reviewed changes

stefanv reviewed May 23, 2024

View reviewed changes

lagru added 🤖 type: Infrastructure CI, packaging, tools and automation and removed 🔧 type: Maintenance Refactoring and maintenance of internals labels May 25, 2024

lagru approved these changes May 25, 2024

View reviewed changes

lagru merged commit 71c41e1 into scikit-image:main Jun 3, 2024
25 checks passed

stefanv added this to the 0.24 milestone Jun 3, 2024

matthewfeickert deleted the ci/add-artifact-attestations branch June 3, 2024 23:02

lagru mentioned this pull request Jun 4, 2024

Create GitHub artifact attestations directly after wheel generation #7432

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Add GitHub artifact attestations to package distribution #7427

CI: Add GitHub artifact attestations to package distribution #7427

matthewfeickert commented May 23, 2024

matthewfeickert May 23, 2024 •

edited

stefanv May 23, 2024

matthewfeickert May 23, 2024

matthewfeickert May 23, 2024

stefanv May 23, 2024

matthewfeickert May 23, 2024

matthewfeickert May 23, 2024

matthewfeickert commented May 23, 2024

stefanv commented May 23, 2024

stefanv left a comment

stefanv May 23, 2024

matthewfeickert May 23, 2024

stefanv May 23, 2024

matthewfeickert commented May 23, 2024

stefanv May 23, 2024

matthewfeickert May 23, 2024

stefanv commented May 23, 2024

lagru left a comment

lagru May 25, 2024

matthewfeickert May 25, 2024

lagru Jun 3, 2024

matthewfeickert commented May 25, 2024 •

edited

lagru commented Jun 4, 2024

		- name: Verify the distribution
		run: pipx run twine check --strict dist/*

	if [ "`which twine`" == "" ]; then
	echo "twine not on path; need to pip install twine?"
	exit 1

CI: Add GitHub artifact attestations to package distribution #7427

CI: Add GitHub artifact attestations to package distribution #7427

Conversation

matthewfeickert commented May 23, 2024

Description

Checklist

Release note

matthewfeickert May 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthewfeickert commented May 23, 2024

stefanv commented May 23, 2024

stefanv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthewfeickert commented May 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stefanv commented May 23, 2024

lagru left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthewfeickert commented May 25, 2024 • edited

lagru commented Jun 4, 2024

matthewfeickert May 23, 2024 •

edited

matthewfeickert commented May 25, 2024 •

edited