Skip to content

Deployment Flow

Jump to bottom
Marcel Boersma edited this page Mar 19, 2024 · 4 revisions

Overview

This document outlines the deployment flow of the FinTorch Python package to PyPI and TestPyPI using GitHub Actions within our repository. The deployment process is triggered by specific events and involves building the package, publishing it to PyPI or TestPyPI, signing the package with Sigstore, and creating a GitHub Release.

Deployment Triggers

  • Main Branch Pushes: Initiates the build process for every push to the main branch, resulting in a release to TestPyPI.
  • Tag Pushes: Triggers the publishing of the package to PyPI, assuming the push is a tag reference.

How to deploy

We employ a strategic branching and tagging workflow to streamline the Python package deployment. First, develop and refine your package code within a dedicated branch. Once thoroughly tested, merge this branch into your 'main' branch. This merge will automatically trigger a deployment to TestPyPI, facilitating rigorous testing in a pre-production environment. If the package satisfies your quality standards, create a version tag locally using the command git tag v0.x.x (substituting your desired version). Subsequently, push this tag to your remote repository with git push --tags. This action initiates a deployment to the production version of PyPI, making your refined package accessible to the broader Python community.

Jobs and Steps

A detailed breakdown of what the GitHub Action pipeline is doing follows.

1. Build Distribution

  • Environment: Ubuntu-latest
  • Key Actions:
    • Checks out the repository code.
    • Sets up the required Python version (3.x).
    • Installs the build module.
    • Builds both a binary wheel and a source tarball of the package.
    • Stores the distribution packages as artifacts.

2. Publish to PyPI

  • Condition: Only on tag pushes to ensure releases are versioned.
  • Environment: Specified PyPI project environment.
  • Key Actions:
    • Downloads the built distribution packages.
    • Publishes the packages to PyPI using a token for authentication.

3. GitHub Release

  • Dependencies: Must occur after successful publication to PyPI.
  • Key Actions:
    • Downloads the built distribution packages.
    • Signs the distributions with Sigstore.
    • Creates a GitHub Release for the tag, including the signed packages and metadata.
    • Uploads artifact signatures and certificates to the GitHub Release.

4. Publish to TestPyPI

  • Environment: Specified TestPyPI project environment.
  • Key Actions:
    • Downloads the built distribution packages.
    • Publishes the packages to TestPyPI, useful for testing and verification before the final PyPI release.

Security and Permissions

  • ID Token: Mandatory for trusted publishing to PyPI and TestPyPI.
  • GitHub Token: Required for creating GitHub Releases and uploading signed packages.

Conclusion

This deployment flow ensures a seamless and secure release process for Python packages, incorporating best practices in build automation, package signing, and release management. By leveraging GitHub Actions, we maintain high standards of code quality and distribution integrity.

References

Clone this wiki locally