Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to pyproject.toml and pypa/gh-action-pypi-publish #6

Closed
simonw opened this issue Jan 16, 2024 · 24 comments
Closed

Upgrade to pyproject.toml and pypa/gh-action-pypi-publish #6

simonw opened this issue Jan 16, 2024 · 24 comments
Labels
enhancement New feature or request

Comments

@simonw
Copy link
Owner

simonw commented Jan 16, 2024

@simonw simonw added the enhancement New feature or request label Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Tried the in-development version of this to build datasette-test:

./LICENSE
./pyproject.toml
./tests
./tests/test_datasette_test.py
./README.md
./setup.py
./.gitignore
./datasette_test
./datasette_test/__init__.py
./.github
./.github/workflows
./.github/workflows/publish.yml
./.github/workflows/test.yml
[project]
name = "datasette-test"
version = "0.1"
description = "Utilities to help write tests for Datasette plugins and applications"
readme = "README.md"
requires-python = ">=3.8"
authors = [{name = "Simon Willison"}]
license = {text = "Apache-2.0"}
classifiers = [
    "License :: OSI Approved :: Apache Software License"
]
dependencies = [

]

[project.urls]
Homepage = "https://github.com/simonw/datasette-test"
Changelog = "https://github.com/simonw/datasette-test/releases"
Issues = "https://github.com/simonw/datasette-test/issues"
CI = "https://github.com/simonw/datasette-test/actions"


[project.optional-dependencies]
test = ["pytest"]

This worked:

pip install -e '.[test]'

The publish.yml looked like this:

name: Publish Python Package

on:
  release:
    types: [created]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
    steps:
    - uses: actions/checkout@v4
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
        cache: pip
        cache-dependency-path: pyproject.toml
    - name: Install dependencies
      run: |
        pip install '.[test]'
    - name: Run tests
      run: |
        pytest
  deploy:
    runs-on: ubuntu-latest
    needs: [test]
    environment: release
    permissions:
      id-token: write
    steps:
    - uses: actions/checkout@v4
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: "3.12"
        cache: pip
        cache-dependency-path: pyproject.toml
    - name: Install dependencies
      run: |
        pip install setuptools wheel build
    - name: Build
      run: |
        python -m build
    - name: Publish
      uses: pypa/gh-action-pypi-publish@release/v1

Compare with this one which I know works: https://github.com/datasette/datasette-build/blob/02e612b667aaa0bc07da24c81023fa6b97d6c270/.github/workflows/publish.yml

Diff here: https://gist.github.com/simonw/968f85a24fead6bf814622b5390d7124/revisions

I think it's going to work. I'm going to risk committing this here, then test it live.

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Annoying: https://github.com/simonw/python-lib/actions/runs/7546633197/job/20544794655

CleanShot 2024-01-16 at 11 37 54@2x

I think I can fix that by manually editing that workflow myself.

simonw added a commit to simonw/python-lib-template-demo that referenced this issue Jan 16, 2024
It can't be updated by a push from GitHub Actions.

simonw/python-lib#6 (comment)
simonw added a commit that referenced this issue Jan 16, 2024
To help debug #6
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Yeah I need to run this manually on my own laptop to help solve this:

CleanShot 2024-01-16 at 11 43 46@2x

simonw added a commit to simonw/python-lib-template-demo that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

git clone git@github.com:simonw/python-lib-template-demo
git clone git@github.com:simonw/python-lib
mv python-lib-template-demo python-lib-template-demo-old
cat python-lib/input-for-demo.txt | cookiecutter python-lib
# Now python-lib-template-demo exists as well
mv python-lib-template-demo-old/.git python-lib-template-demo
cd python-lib-template-demo
git diff
# Looks good
git commit -a -m 'Updates'
git push

simonw added a commit to simonw/python-lib-template-demo that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

That seemed to work - here's the new example output, which passes its tests: https://github.com/simonw/python-lib-template-demo/tree/8ca44689

And here's the full diff simonw/python-lib-template-demo@f417323...8ca4468

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Next test: use https://github.com/new?template_name=python-lib-template-repository&template_owner=simonw to create a datasette-test package and then ship it!

simonw referenced this issue in datasette/datasette-test Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

The test.yml workflow in that is completely wrong - full of setup.py references: https://github.com/datasette/datasette-test/blob/8d5f8262dc3a88f3c6d97f0cef3b55264cabc695/.github/workflows/test.yml

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

I'm really confused. https://github.com/simonw/python-lib/blob/main/%7B%7Bcookiecutter.hyphenated%7D%7D/.github/workflows/test.yml has the correct stuff in it. Why did datasette-test get the old version?

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Oh! That's because I needed to update these specific files:

That's my workaround for the problem where you can't push an update to a workflow - I include copies of them in the template repository itself.

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Fixing that like so, after generating a fresh output in /tmp:

cd python-lib-template-repository
cp /tmp/datasette-test/.github/workflows/test.yml .github/workflows/test.yml
cp /tmp/datasette-test/.github/workflows/publish.yml .github/workflows/publish.yml 

simonw added a commit to simonw/python-lib-template-repository that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

I'm going to try generating a repo from that manually: https://github.com/new?template_name=python-lib-template-repository&template_owner=simonw

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Created: https://github.com/simonw/python-lib-issue-6

I'll copy the workflows from that into https://github.com/datasette/datasette-test/

simonw added a commit to datasette/datasette-test that referenced this issue Jan 16, 2024
simonw added a commit to datasette/datasette-test that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

I'm going to ship 0.1 of datasette/datasette-test to see if the publish workflow works.

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Trying that with a Trusted Publisher:

CleanShot 2024-01-16 at 12 37 45@2x

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

There's one sharp edge left: the tests fail the first time the repository is cloned from the template: simonw/python-lib-issue-6@54887b2

https://github.com/simonw/python-lib-issue-6/actions/runs/7547200692/job/20546630262 failed:

CleanShot 2024-01-16 at 12 41 53@2x

simonw added a commit to simonw/python-lib-template-repository that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

CleanShot 2024-01-16 at 12 46 45@2x

It doesn't like that there are no tests.

simonw added a commit to simonw/python-lib-template-repository that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

One more test: https://github.com/simonw/python-lib-issues-6-demo-2

Tests now pass on initial commit: simonw/python-lib-issues-6-demo-2@73d216e

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

The setup.py did show some warnings: https://github.com/simonw/python-lib-issues-6-demo-2/actions/runs/7547373090

CleanShot 2024-01-16 at 12 52 24@2x

simonw added a commit to simonw/python-lib-template-repository that referenced this issue Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

I'll use that to create a new test repo, check for no errors, then delete that repo.

CleanShot 2024-01-16 at 12 54 18@2x

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Yes, that worked - I added whitespace to the tests/*.py file too to make sure the tests passed.

@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

This is done!

One last thing: I'm going to update the documentation on https://github.com/simonw/python-lib to remind me how to update the template in the future.

@simonw simonw closed this as completed in 74ceb1b Jan 16, 2024
@simonw
Copy link
Owner Author

simonw commented Jan 16, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant