Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a workflow to release and sign wheels #22

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions .github/workflows/cd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: CD

on:
workflow_dispatch:
inputs:
version:
description: 'Version to build Zig wheels for'
required: true
default: 'latest'
suffix:
description: >
Suffix to append to the version in the wheel filename, useful for dev versions and version specifiers
required: false
default: ''
platforms:
description: >
Comma-separated values of platforms to build wheels for
required: false
default: 'x86_64-windows,x86-windows,x86_64-macos,aarch64-macos,i386-linux,x86-linux,x86_64-linux,aarch64-linux,armv7a-linux'
Comment on lines +15 to +19
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be overkill as well – I was just experimenting with this locally, so it can be good if you don't wish to build too many wheels at once, but if a wheel or architecture has to be backported for a particular version, it can be useful.

push_to_pypi:
description: >
Whether to push the built wheels to PyPI. Can be 'true' or 'false', defaults to 'false'.
required: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
required: false
required: true

It might make sense to make this a required input so that the user is aware of what is being input here.

default: 'false'

jobs:
build_wheels:
name: Build wheels
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: '3.x'

- name: Install dependencies
run: |
python -m pip install .

- name: Build wheels for all platforms
shell: bash
run: |
platforms=${{ github.event.inputs.platforms }}
IFS=',' read -r -a platform_array <<< "$platforms"
for platform in "${platform_array[@]}"; do
python make_wheels.py \
--version ${{ github.event.inputs.version }} \
--suffix ${{ github.event.inputs.suffix }} \
--platform "$platform"
done

- name: Upload wheel artifacts
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: zig-wheels
path: dist/*.whl

deploy_wheels:
name: Deploy wheels
needs: [build_wheels]
environment: pypi
runs-on: ubuntu-latest
Comment on lines +60 to +62
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This environment will have to be created in the repository settings, as noted in the PR description. It will trigger the workflow, but the deployment will pass only if it gets approved, and the workflow won't be triggered until so.

permissions:
# Required by
# 1. OIDC to publish to PyPI, and
# 2. Sigstore to sign artifacts
id-token: write
if: >-
github.event_name == 'workflow_dispatch' &&
github.event.inputs.push_to_pypi == 'true' &&
github.repository == 'ziglang/zig-pypi'
Copy link
Contributor Author

@agriyakhetarpal agriyakhetarpal Apr 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move this to the context of the step that uploads to PyPI instead of the entire job, because we can have scenarios where we want to build a wheel, upload to PyPI manually (locally) using twine, but also have the Sigstore files associated with it as marker that the upload has been signed.

This can be useful in cases like #18 where PyPI already has Zig wheels for some platforms for a particular version, and one needs to build extra wheels for two versions, so instead of doing so locally, one can do it from here by triggering the workflow twice, and push the Sigstore files somewhere later to establish their permanence.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in cf9845e.

steps:
- name: Download wheel artifacts
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
path: dist/
merge-multiple: true

- name: Publish wheels to PyPI
uses: pypa/gh-action-pypi-publish@81e9d935c883d0b210363ab89cf05f3894778450 # v1.8.14
with:
packages-dir: dist/

- name: Sign artifacts with Sigstore
uses: sigstore/gh-action-sigstore-python@61f6a500bbfdd9a2a339cf033e5421951fbc1cd2 # v2.1.1
with:
inputs: >-
./dist/*.whl

- name: Upload signed artifacts and signature files
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
# This will contain not only the wheels but also the signature files
# generated by the Sigstore step
path: dist/*
Comment on lines +84 to +95
Copy link
Contributor Author

@agriyakhetarpal agriyakhetarpal Apr 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The artifact retention policy for GitHub Actions artifacts, which are, in this case, the files generated from Sigstore, is set to default to 90 days from the completion of the workflow (not sure what the upper limit is). Having a place to put them permanently would serve some part of the reproducibility issues discussed. The available options are:

  1. Publish them to GitHub Releases: and releases require git tags, which is something that was objected to and we agreed not to do
  2. Upload them in a folder in the repository in a manual commit after downloading them (not the wheels though, of course)
  3. Upload them to a pinned issue and store them in the issue comments
  4. and so on.

I think the second one is the best option at this time, because users can download and use those to verify them.