-
Notifications
You must be signed in to change notification settings - Fork 12
Fix: update release workflow to follow pypa and pypi guidelines #73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 11 commits
81ed956
f7ae35d
878094c
cf61545
aa4c259
ddc5597
48f3cc2
375b29e
471236b
00884d7
ad839c5
62d7ef4
022b6c7
4d535ad
6dce9fe
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# pyOpenSci packaging template Changelog | ||
|
||
## [Unreleased] | ||
|
||
* Update release workflow to follow PyPA / PyPI recommended practices (@lwasser, #48) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# Contributing to the pyOpenSci Python package template | ||
|
||
To work on the template locally, you can call the copier template directly. Note that by default, `copier` uses the latest tag in your commit history. To ensure it uses the latest commit on your current active branch use: | ||
|
||
`copier copy -r HEAD /path/to/your/template destination-dir` | ||
|
||
If you want to test it against the latest tag in your local commit history, you can use: | ||
|
||
`copier copy /path/to/your/template destination-dir` | ||
|
||
## Run the tests | ||
|
||
You can use Hatch to run all of the tests for the template: | ||
|
||
`hatch run test:run` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,53 +1,77 @@ | ||
name: CD | ||
name: Publish to PyPI | ||
|
||
on: | ||
push: | ||
tags: | ||
- 'v?[0-9]+.[0-9]+.[0-9]+' | ||
- 'v?[0-9]+.[0-9]+.[0-9]+(a|b|rc|post|dev)[0-9]+' | ||
release: | ||
types: [published] | ||
|
||
jobs: | ||
prerequisites: | ||
uses: ./.github/workflows/test.yml | ||
|
||
release: | ||
# Setup build separate from publish for added security | ||
# See https://github.com/pypa/gh-action-pypi-publish/issues/217#issuecomment-1965727093 | ||
build: | ||
needs: [prerequisites] | ||
willingc marked this conversation as resolved.
Show resolved
Hide resolved
|
||
strategy: | ||
matrix: | ||
os: [ubuntu-latest] | ||
python-version: ["3.12"] | ||
runs-on: ${{ matrix.os }} | ||
permissions: | ||
# Write permissions are needed to create OIDC tokens. | ||
id-token: write | ||
# Write permissions are needed to make GitHub releases. | ||
contents: write | ||
|
||
runs-on: ubuntu-latest | ||
# Environment is encouraged for increased security | ||
environment: build | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a new one for me, so I'm curious. Could you please share how it works? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh! @agriyakhetarpal environments allow you to setup a trusted connection with pypi. This is the build environment, so we don't need to worry too much about security here in terms of PYPI. so anyone can run the build step in theory. but by having a publish environment, we can control who can trigger that step of the build. (if that is what you're asking) . here is a bit more about this There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, I'm aware of trusted publishing and environments, thanks! My question was more around the need for this There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh gosh my apologies. i suspected I was telling you something you already knew. 🙈 This is a great point. I don't know the answer to your question!! But i suspect this is ME introducing complexity. i just checked the tutorials and there is a build step but not an additional environment. Shall we remove this environment as you are correct it's not required. we need the job but not an environment AFAIK . There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
No worries at all! 😄
Thank you for dropping it!
lwasser marked this conversation as resolved.
Show resolved
Hide resolved
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
with: | ||
# This fetch element is only important if you are use SCM based | ||
# versioning (that looks at git tags to gather the version) | ||
fetch-depth: 100 | ||
# Need the tags so that setuptools-scm can form a valid version number | ||
- name: Fetch git tags | ||
run: git fetch origin 'refs/tags/*:refs/tags/*' | ||
lwasser marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- name: Install hatch | ||
uses: pypa/hatch@install | ||
- name: Setup Python | ||
uses: actions/setup-python@v5 | ||
with: | ||
# You can modify what version of Python you want to use for your release | ||
python-version: "3.11" | ||
|
||
- name: Build package | ||
run: hatch build | ||
# Security recommends we should pin deps. Should we pin the workflow version? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, please. I'll recommend that all GitHub Actions—whether by GitHub as an actor or external ones available on the marketplace—should be pinned to their commit hashes, at least for a workflow concerning a release. There is a case to be made about how it makes things a bit more challenging for new contributors/packagers, but I think supply-chain security shouldn't be at the end of a compromise. There are tools such as There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. great. So, looking at the action here Ofek suggests that too - we'd use this - this hash is almost the most recent but the most recent is just a readme update.
We should do that as part of this repo's maintenance, but also, we'll want to tell users to update this periodically, right? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Updating it weekly for our maintenance could be a good cadence; what do you think? It would depend on the amount of activity we have on this template. If we were to ask our users to enable Dependabot in the GitHub settings and inherit the same settings, they should also get the same updates. However, I'm not sure whether it's too much to ask for beginners (😕). As long as it is pinned for a start, we can proceed with a small guide on how to keep this workflow maintained, where we can describe this more (or link to an external guide if that's alright and vetted by us?). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @agriyakhetarpal this is awesome. We, I think, have Depdendabot set up in the org, I think, but I didn't configure it. This is a basic question but would it help update things like this version? IE could we use it in this repo? My gut tells me we won't have the bandwidth to perform weekly updates here unless someone wants to take that one! But perhaps monthly or quarterly would work. I am not sure what is best, however and would love suggestions. also if you were interested in helping us maintain here, that would be awesome (only if you have bandwidth!) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Oh, I notice that here are a few moving parts, here. As this file is not a workflow of its own but a template that will be generated and moved into That means that we have two possible scenarios:
The first one is surely not ideal, so we should explore if Dependabot will choose to be the star here or not 😉
I agree with you. Monthly updates would be nicer – and users can always bump it to daily, or weekly, and so on, based on their bandwidth.
I'd love to! I've also helped out with the development and maintenance of especially other There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok wonderful!! @agriyakhetarpal i'm going to create an issue for thiw topic specifically as it's beyon the scope of this pr and merging this would be nice!! BUT it's an important topic. So let's move your comment above to an issue. Also i'd love to add you to our packaging council team here at pyOpenSci which will give ou more superpowers in our repos!! I"ll do that now. So happy to have you here and also happy that @Midnighter is able to help me undstand some of the technical points and updates that you have so carefully pointed out and fixed!! thank you both! |
||
- name: Install hatch | ||
uses: pypa/hatch@a3c83ab3d481fbc2dc91dd0088628817488dd1d5 | ||
|
||
# We rely on a trusted publisher configuration being present on PyPI, | ||
# see https://docs.pypi.org/trusted-publishers/. | ||
- name: Publish to PyPI | ||
uses: pypa/gh-action-pypi-publish@release/v1 | ||
- name: Build package using Hatch | ||
run: | | ||
hatch build | ||
echo "" | ||
echo "Generated files:" | ||
ls -lh dist/ | ||
|
||
- name: GH release | ||
uses: softprops/action-gh-release@v2 | ||
with: | ||
body: > | ||
Please see | ||
https://github.com/${{ github.repository }}/blob/${{ github.ref_name }}/CHANGELOG.md | ||
for the full release notes. | ||
draft: false | ||
prerelease: false | ||
# Store an artifact of the build to use in the publish step below | ||
- name: Store the distribution packages | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: python-package-distributions | ||
path: dist/ | ||
lwasser marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if-no-files-found: error | ||
publish: | ||
name: >- | ||
Publish Python 🐍 distribution 📦 to PyPI | ||
# Modify the repo name below to be your project's repo name. | ||
if: github.repository_owner == "{{ username }}" | ||
needs: | ||
- build | ||
runs-on: ubuntu-latest | ||
# Environment required here for trusted publisher | ||
environment: | ||
name: pypi | ||
# Modify the url to be the name of your package | ||
url: https://pypi.org/p/${{ package_name }} | ||
permissions: | ||
id-token: write # this permission is mandatory for PyPI publishing | ||
steps: | ||
- name: Download dists | ||
uses: actions/download-artifact@v4 | ||
with: | ||
name: python-package-distributions | ||
path: dist/ | ||
lwasser marked this conversation as resolved.
Show resolved
Hide resolved
|
||
merge-multiple: true | ||
- name: Publish package to PyPI | ||
# Only publish to real PyPI on release | ||
if: github.event_name == 'release' | ||
lwasser marked this conversation as resolved.
Show resolved
Hide resolved
|
||
uses: pypa/gh-action-pypi-publish@release/v1 |
Uh oh!
There was an error while loading. Please reload this page.