diff --git a/README.md b/README.md index 449db9f..f5eed6e 100644 --- a/README.md +++ b/README.md @@ -2,26 +2,20 @@ [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![CodeQL](https://github.com/joergschultzelutter/pypi-publish-workflow/actions/workflows/codeql.yml/badge.svg)](https://github.com/joergschultzelutter/pypi-publish-workflow/actions/workflows/codeql.yml) -This is a __Github Actions__ workflow for automatic publications to PyPi. Version data from a python file is extracted and then used by the PyPi setup process which will publish the package to PyPi Test and Prod. +This is a __Github Actions__ workflow for automatic publications to PyPi. Version data from a python file is extracted and then used by the PyPi setup process which will publish the package to PyPi Test and Prod, following PyPi's [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) model. The workflow will only be triggered for the publication of new repo releases / prereleases for the 'master' repo branch. -## Installation instructions - -### Setup Github Secrets - -- Create token secrets for both [PyPi Test](https://test.pypi.org/) and [PyPi Prod](https://www.pypi.org/) (``Account Settings`` > ``API Tokens`` > ``Add API token``). -- In your Github project, goto ``Settings`` > ``Secrets and Variables`` > ``Actions`` -- Create two `Secrets` keys (`New repository secret`) named ``TEST_PYPI_API_TOKEN`` and ``PROD_PYPI_API_TOKEN`` and assign the previously created token secrets to these keys - -### Overview on config files +## Overview on config files This repo contains three files that you may need to amend and copy to your Github repository: - ``setup.py``: this is a regular Python ``setup.py`` file; amend the file content with your package information and then save the file in your repo's root directory - ``publish-to-pypi.yml``: Edit this file, amend the configuration settings (see next chapter) and then save the file in your repo's Github Actions directory (``.github/workflows``). You may also need to activate the new workflow once you have installed it - see [documentation on Github](https://docs.github.com/en/actions). -### `publish-to-pypi.yml` configuration +## Configuration file instructions + +### `publish-to-pypi.yml` Open the file. You will notice a section which looks like this: @@ -98,6 +92,28 @@ Necessary steps for a manual usage: - open `setup.py` and assign a version number to the `GITHUB_PROGRAM_VERSION` variable - `pip install git+https://github.com/my-repository-name@my-branch#egg=my-package-name` +## Installation instructions + +The workflow uses PyPi's [Trusted Publisher](https://docs.pypi.org/trusted-publishers/) model. For a new project on PyPi, follow [these instructions](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) for setting up a trusted publisher. For an existing project which you may want to migrate from a secret-based workflow to the new trusted workflows, use [these instructions](https://docs.pypi.org/trusted-publishers/adding-a-publisher/). + +### Step 1: set up a Github environment + +- In your GitHub project, go to **Settings > Environments** and create a new environment called `pypi`. +- Configure `Required reviewers` or other settings, if necessary. You do NOT need to configure any secrets here. + +### Step 2: Deploy the workflow and the setup file + +- You need to configure the files prior to deployment, see previous chapter **Configuration file instructions** +- `setup.py` goes into your repository's root directory +- `publish-to-pypi.yml` goes into your repository's `.github/workflows` directory (or add as a new GitHub action) + +### Step 3: Trusted Publisher Setup + +- Log on to your PyPi Test & Prod accounts +- Follow the instructions on how to set up a [Trusted Publisher](https://docs.pypi.org/trusted-publishers/) on both Test and Prod environments: + - Set the `Workflow Name` to `publish-to-pypi.yml` + - Set the `Environment` to `pypi` + ## Running the Github Action This Github action will do the following __whenever a new release/pre-release is published for the 'master' branch__: @@ -105,13 +121,13 @@ This Github action will do the following __whenever a new release/pre-release is - Read the Python file and extract the version information, based on the given Regex. Abort job if no match was found. - Check if the Github ``ref_type`` has the value ``tag``. This is only the case when you drafted a new release. Otherwise, this value is likely set to ``master``. Abort job in case of a mismatch. - Check if the Github ``ref_name`` is equal to the extracted version from you Python file. Abort job in case of a mismatch. This will prevent issues where there is a mismatch between your Github release version (tag) and the one in the Python file. -- Build the PyPi package. Deploy it to PyPi Test and (if successful AND not a pre-release) PyPi Prod. +- Build the PyPi package. Deploy it to PyPi Test and (if successful AND not a pre-release) PyPi Prod. Note: This is done as a separate workflow step, see [this issue](https://github.com/pypa/gh-action-pypi-publish/issues/319) for technical details. This job will be triggered for releases AND prereleases in 'created' state (read: you tag a (pre)release in Github). Releases will be pushed to both PyPi Test and Prod whereas prereleases will only be pushed to PyPi Test. ## Test your work flow -In case you want to become acquainted with this work flow: The safest way to test the work flow is to create both Github secret entries ``TEST_PYPI_API_TOKEN`` and ``PROD_PYPI_API_TOKEN`` but assign an invalid token to them. When you run the workflow for a new 'master' branch prerelease, the job will try to push it to PyPi Test and will fail because of the invalid token. +- Publish your package as a prerelease. This should deploy your code only to PyPi Test. ## Workflow A basic workflow diagram of this Github Action can be found [here](docs/workflow.jpg) diff --git a/publish-to-pypi.yml b/publish-to-pypi.yml index 6e3d387..ff87433 100644 --- a/publish-to-pypi.yml +++ b/publish-to-pypi.yml @@ -84,6 +84,7 @@ jobs: # to next Github Actions job in line. get-python-version-info: runs-on: ubuntu-latest + environment: pypi permissions: write-all # Output which is passed to the PyPi publication job @@ -150,12 +151,21 @@ jobs: # # BEGIN of Job 2 # - # This section will create the PyPi package and first deploy it to PyPi test. - # If successful, it will also try to issue a PyPi Prod deployment afterwards + # This section will create the PyPi package and deploy it to PyPi test. + # in case a pre-release was selected # - deploy-to-pypi: + # NOTE: When using PyPi's 'Trusted Publishing", one cannot use the same + # OIDC token for both Test and Prod - otherwise, error + # "Attestation generation failure: The following distributions already have publish attestations" + # will be thrown. See https://github.com/pypa/gh-action-pypi-publish/issues/319 for details + # This is the only reason for Test and Prod not sharing the same work flow + # + deploy-to-pypi-test: runs-on: ubuntu-latest + environment: pypi needs: get-python-version-info + permissions: + id-token: write steps: @@ -187,17 +197,65 @@ jobs: if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') uses: pypa/gh-action-pypi-publish@release/v1 with: - user: __token__ - password: ${{ secrets.TEST_PYPI_API_TOKEN }} + #user: __token__ + #password: ${{ secrets.TEST_PYPI_API_TOKEN }} repository-url: https://test.pypi.org/legacy/ + # + # END of Job 2 + # + # + # BEGIN of Job 3 + # + # This section will create the PyPi package and fdeploy it to PyPi Prod. + # + # NOTE: When using PyPi's 'Trusted Publishing", one cannot use the same + # OIDC token for both Test and Prod - otherwise, error + # "Attestation generation failure: The following distributions already have publish attestations" + # will be thrown. See https://github.com/pypa/gh-action-pypi-publish/issues/319 for details + # This is the only reason for Test and Prod not sharing the same work flow + # + deploy-to-pypi-prod: + runs-on: ubuntu-latest + environment: pypi + needs: [get-python-version-info, deploy-to-pypi-test] + permissions: + id-token: write + + steps: + + - uses: actions/checkout@v5 + # Set up Python environment + - name: Set up Python + if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease + uses: actions/setup-python@v6 + with: + python-version: '${{ env.PYTHON_VERSION }}' + + # Install all dependencies + - name: Install dependencies + if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease + run: | + python -m pip install --upgrade pip + pip install build + + # Export the program version; content will be picked up by the setup.py script + - name: Export program version + if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease + run: echo GITHUB_PROGRAM_VERSION='${{ needs.get-python-version-info.outputs.my-program-version }}' >> $GITHUB_ENV + + # Build the package. The export MUST be part of THIS step + # Otherwise, the Python setup job will not see this information + - name: Build package + if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease + run: python -m build + # Publish everything to Prod PyPi but only if it is not a prerelease - name: Publish package to Prod PyPi if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease uses: pypa/gh-action-pypi-publish@release/v1 - with: - user: __token__ - password: ${{ secrets.PROD_PYPI_API_TOKEN }} - # - # END of Job 2 + #with: + #user: __token__ + #password: ${{ secrets.PROD_PYPI_API_TOKEN }} # + # END of Job 3