Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for trusted publishers. #73

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions .github/workflows/main.yml → .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
name: build
name: CI

on: [push, pull_request]

env:
PYTEST_ADDOPTS: "--color=yes"
FORCE_COLOR: "1" # Make tools pretty.
TOX_TESTENV_PASSENV: FORCE_COLOR
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_NO_PYTHON_VERSION_WARNING: "1"

jobs:
test:
Expand Down Expand Up @@ -101,35 +104,36 @@ jobs:
- name: Run linters
run: poetry run invoke lint


deploy:
name: Deploy
prep-release:
name: Release preparations
environment: Deployment
needs: [test, lint]
runs-on: ubuntu-latest
if: ${{ github.ref=='refs/heads/main' && github.event_name!='pull_request' }}

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Setup Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: "3.8"
- name: Check release
id: check_release
run: |
python -m pip install --upgrade pip
python -m pip install poetry githubrelease httpx==0.16.1 autopub
echo "##[set-output name=release;]$(autopub check)"
- name: Publish
python -m pip install poetry autopub[github]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the github extra works just fine?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I believe the github extra should work fine. There was a point at which I pinned httpx in order to work around an issue with githubrelease, but I imagine that has since been resolved. I'd rather take the approach you have here and then re-pin httpx if and when needed. 👍

echo "release=$(autopub check)" >> "$GITHUB_OUTPUT"
EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
echo "release<<$EOF" >> "$GITHUB_OUTPUT"
Copy link
Collaborator Author

@apollo13 apollo13 Jun 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set-ouput is deprecated — I have switched it over to multiline strings since we will never know how much autopub check will output… If autopub check were to write into $GITHUB_OUTPUT/STATE on its own, we wouldn't need to handle this extra. I mean it has special handling for CircleCI also… https://github.com/autopub/autopub/blob/8d41aed1ca9ff0aa197a7850eb7d4314a96a1206/autopub/check_release.py#L13

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention was always for AutoPub to internally handle as much of the machinery as possible, so I agree that AutoPub should be enhanced to obviate the need to handle this in GitHub Actions. 👍

autopub check >> "$GITHUB_OUTPUT"
echo "$EOF" >> "$GITHUB_OUTPUT"
- name: Tag & Create GH release
if: ${{ steps.check_release.outputs.release=='' }}
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
git remote set-url origin https://$GITHUB_TOKEN@github.com/${{ github.repository }}
autopub prepare
poetry build
autopub build
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to set the build-system in pyproject.toml for that to work since autopub does not seem to detect that?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. That's odd. Even without adding an explicit key+value for tool.autopub.build-system, AutoPub should indeed detect it automatically from build-system.requires, as you can see here: https://github.com/autopub/autopub/blob/8d41aed1ca9ff0aa197a7850eb7d4314a96a1206/autopub/base.py#L94-L96

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I have some cleanups for it there: autopub/autopub#38 -- requires is a list so the in would only check for requires = ['poetry']. Using the build_backend instead with a bit of fuzzy matching seems to work though.

autopub commit
autopub githubrelease
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like that the githubrelease upload has the packages from autopub build and the pypi upload builds it's own packages. I'd also love to use build-and-inspect-python-package from the pypi-package workflow to be able to inspect the outputs properly before. I guess I might move autopub githubrelease into the other workflow and run the other workflow on tags only? This would still result in a tag being pushed but we could then only run autopub githubrelease & pypi upload after we checked the output from build-and-inspect-python-package manually?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — this seems like a more sensible approach. 👍

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arg, the tags are not created by autopub commit but by autopub githubrelease. Honestly autopub is really trying to make this hard here. But I am somewhat out of ideas :/

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tag is created as part of the githubrelease step because the latter needs to know the tag in order to create the release. It might be feasible to separate the tagging step into a separate command (autopub tag), but then we would need some way to maintain "state" such that the tag info could be passed to the githubrelease step.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm the state could be the tag (?) of the current commit?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In an effort to find a more efficient place for us to discuss this topic, I created an AutoPub issue along with a proposal. What do you think? autopub/autopub#39

poetry publish -u __token__ -p $PYPI_PASSWORD
42 changes: 42 additions & 0 deletions .github/workflows/pypi-package.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
name: Build & upload PyPI package

on:
release:
types:
- published

jobs:
build-package:
name: Build & verify package
runs-on: ubuntu-latest
permissions:
contents: read

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0

- uses: hynek/build-and-inspect-python-package@v1

release-pypi:
name: Publish released package to pypi.org
environment: release-pypi # TODO: how to name the env
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also Deployment like we have now?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deployment seems like a sensible choice to me. 👍

if: github.event.action == 'published'
runs-on: ubuntu-latest
needs: build-package
permissions:
id-token: write

steps:
- name: Download packages built by build-and-inspect-python-package
uses: actions/download-artifact@v3
with:
name: Packages
path: dist

- name: Upload package to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/ # TODO: Move over to actual PyPI
10 changes: 4 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,6 @@ classifiers = [
"Intended Audience :: Developers",
"License :: OSI Approved :: BSD License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Topic :: Security",
"Topic :: Security :: Cryptography",
"Topic :: Software Development :: Libraries :: Python Modules",
Expand Down Expand Up @@ -60,6 +54,7 @@ Werkzeug = "^2.0"

[tool.autopub]
project-name = "Kagi"
build-system = "poetry"
git-username = "botpub"
git-email = "botpub@autopub.rocks"
append-github-contributor = true
Expand Down Expand Up @@ -92,3 +87,6 @@ DJANGO_SETTINGS_MODULE = 'testproj.settings'
[tool.flake8]
ignore = ['E203', 'E501', 'W503']
max-line-length = 88

[tool.check-wheel-contents]
ignore = ['W004']
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Required because Django migrations are not valid module names.