Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a guide about publishing dists via GH Actions #647

Merged
Merged
Changes from 10 commits
Commits
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -0,0 +1,40 @@
name: Publish Python 🐍 distribution package 📦 to PyPIs
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
name: Publish Python 🐍 distribution package 📦 to PyPIs
name: Publish Python distributions to PyPI and TestPyPI

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

You don't like emojis?

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member
Suggested change
name: Publish Python 🐍 distribution package 📦 to PyPIs
name: Publish Python 🐍 distributions 📦 to PyPI and TestPyPI

on: push

jobs:
build-n-publish:
name: Build and publish Python 🐍 dist 📦 to PyPIs
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
name: Build and publish Python 🐍 dist 📦 to PyPIs
name: Build and publish Python distributions to PyPI and TestPyPI

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member
Suggested change
name: Build and publish Python 🐍 dist 📦 to PyPIs
name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
runs-on: ubuntu-18.04

steps:
- uses: actions/checkout@master
- name: Set up Python 3.7
uses: actions/setup-python@v1
with:
version: 3.7
- name: Install pep517
run: >-
python -m
pip install
pep517
--user
- name: Build a binary wheel and a source tarball
run: >-
python -m
pep517.build
--source
--binary
--out-dir dist/
.
# Actualy publish to PyPIs
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
# Actualy publish to PyPIs
# Actually publish to PyPI/TestPyPI
- name: Publish 📦 to Test PyPI
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
- name: Publish 📦 to Test PyPI
- name: Publish distribution to TestPyPI

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member
Suggested change
- name: Publish 📦 to Test PyPI
- name: Publish distribution 📦 to Test PyPI
uses: pypa/gh-action-pypi-publish@master

This comment has been minimized.

Copy link
@hugovk

hugovk Sep 18, 2019

Contributor

Once there's a release, include the version number here.

We strongly recommend that you include the version of the action you are using by specifying a Git ref, SHA, or Docker tag number. If you don't specify a version, it could break your workflows or cause unexpected behavior when the action owner publishes an update.

  • Using the commit SHA of a released action version is the safest for stability and security.
  • Using the specific major action version allows you to receive critical fixes and security patches while still maintaining compatibility. It also assures that your workflow should still work.
  • Using the master branch of an action may be convenient, but if someone releases a new major version with a breaking change, your workflow could break.

https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstepsuses

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 18, 2019

Author Member

Sure, I'd also mention it somewhere in the README of the action itself.

with:
password: ${{ secrets.test_pypi_password }}

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
password: ${{ secrets.test_pypi_password }}
password: ${{ secrets.testpypi_password }}
repository_url: https://test.pypi.org/legacy/
- name: Publish 📦 to production PyPI
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
- name: Publish 📦 to production PyPI
- name: Publish distribution to PyPI

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member
Suggested change
- name: Publish 📦 to production PyPI
- name: Publish distribution 📦 to PyPI
if: startsWith(github.event.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.pypi_password }}
@@ -29,3 +29,4 @@ introduction to packaging, see :doc:`/tutorials/index`.
migrating-to-pypi-org
using-testpypi
making-a-pypi-friendly-readme
publishing-package-distribution-releases-using-github-actions-ci-cd-workflows
@@ -0,0 +1,148 @@
=============================================================================
Publishing package distribution releases using GitHub Actions CI/CD workflows
=============================================================================

`GitHub Actions CI/CD`_ allows you to run a series of commands
whenever an event occurs on the GitHub platform. One
popular choice is having a workflow that's triggered by a
``push`` event.
This guide shows you how to publish a Python distribution
package whenever a tagged commit is pushed.
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
package whenever a tagged commit is pushed.
whenever a tagged commit is pushed.

.. attention::

This guide *assumes* that you already have a project that
you know how to build dists for and *it lives on GitHub*.
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
you know how to build dists for and *it lives on GitHub*.
you know how to build distributions for and *it lives on GitHub*.

.. warning::

At the time of writing, `GitHub Actions CI/CD`_
is in public beta. If you don't have it enabled,
you should `join the waitlist`_ to gain access.

GitHub Actions will be generally available on November
13th, 2019.
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member

Seems like this is already more or less true: https://twitter.com/natfriedman/status/1174158823877042177?s=20

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

Right, he announced this in the yesterday's stream along with the security announcements. But he also mentioned this information in the previous stream, about a month ago.

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

Are you suggesting that we remove this note? Or is it fine to keep it till the given date?

This comment has been minimized.

Copy link
@hugovk

hugovk Sep 19, 2019

Contributor

I think the note could be removed now, basically you're now in the beta immediately without any waitlist 👍

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

Should I only delete this sentence or the whole warning?



Saving credentials on GitHub
============================

In this guide, we'll demonstrate uploading to both production
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
In this guide, we'll demonstrate uploading to both production
In this guide, we'll demonstrate uploading to both
PyPI and Test PyPI meaning that we'll have two separate sets
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
PyPI and Test PyPI meaning that we'll have two separate sets
PyPI and TestPyPI, meaning that we'll have two separate sets
of creds. And we'll need to save them in the GitHub repo
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
of creds. And we'll need to save them in the GitHub repo
of credentials. And we'll need to save them in the GitHub repository
settings.

Let's begin! 🚀

1. Go to https://pypi.org/manage/account/#api-tokens and

This comment has been minimized.

Copy link
@hugovk

hugovk Sep 16, 2019

Contributor

Maybe put the Test PyPI instructions before production PyPI?

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 16, 2019

Author Member

I think users usually have an account on prod, but rarely on test...

create a new `API token`_. If you have the project on PyPI
already, limit the token scope to just that project.
You can call it something like
``GitHub Actions CI/CD — project-org/project-repo``
in order for it to be easily distinguishable in the token
list.
**Don't close the page just yet — you won't see that token
again.**
2. In a separate browser tab or window, go to the ``Settings``
tab of your target repository and then click on `Secrets`_
in the left sidebar.
3. Create a new secret called ``pypi_password`` and copy-paste
the token from the fist step.
4. Now, go to https://test.pypi.org/manage/account/#api-tokens
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member

Maybe add a note here that you might need to create a TestPyPI account? The fact that PyPI accounts don't work on TestPyPI often confuses users.

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

Fair enough, I'll add this info.

and repeat the steps. Save that Test PyPI token on GitHub
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
and repeat the steps. Save that Test PyPI token on GitHub
and repeat the steps. Save that TestPyPI token on GitHub
as ``test_pypi_password``.

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
as ``test_pypi_password``.
as ``testpypi_password``.

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

I think that it's better readable to maintain snake_case here.

This comment has been minimized.

Copy link
@hugovk

hugovk Sep 19, 2019

Contributor

Is testpypi_password the snake_case version of TestPyPI (no space)?

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

TestPyPI is CamelCase version and it's clearly visible that test is a separate word, even though it's spelled together in the capitalized version. So I consider test_pypi_password a proper snake_case version.



Creating a workflow definition
==============================

GitHub CI/CD workflows are declared in YAML files stored under
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
GitHub CI/CD workflows are declared in YAML files stored under
GitHub CI/CD workflows are declared in YAML files stored in the
``.github/workflows/`` of your repository.
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
``.github/workflows/`` of your repository.
``.github/workflows/`` directory of your repository.

Let's create a ``.github/workflows/publish-to-test-pypi.yml``
file.

Start it with a meaningful name and define the event that
should make GitHub run this workflow:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

I think, it's better readable to keep the separation.

:language: yaml
:end-before: jobs:


Defining a workflow job environment
===================================

Now, let's add initial setup for our job. It's a process that
will execute commands that we'll define later.
In this guide, we'll use Ubuntu 18.04:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml
:language: yaml
:start-after: on:
:end-before: steps:


Checking out the project and building dists
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
Checking out the project and building dists
Checking out the project and building distributions
===========================================
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
===========================================
===================================================

Then, add the following under the ``build-n-publish`` section:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml
:language: yaml
:start-after: runs-on:
:end-before: Install pep517

This will download your repository into the CI runner and then
install and activate Python 3.7.

And now we can build dists from source. In this example, we'll
use ``pep517`` package, assuming that your project has a
``pyproject.toml`` properly set up (see
:pep:`517`/:pep:`518`).

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member

Maybe this would be a good place to explain what the pep517 package is? Most users haven't seen this before.

This comment has been minimized.

Copy link
@webknjaz

webknjaz Sep 19, 2019

Author Member

Yeah, I wasn't sure about this. AFAIK it's not really intended for end-users but it was an easy way to inject "some way" of building dists as I'm trying not to focus on building in this guide and point users to their favorite building methods...


.. tip::

You can use any other method for building dists as long as
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
You can use any other method for building dists as long as
You can use any other method for building distributions as long as
it produces ready-to-upload artifacts saved into the
``dist/`` folder.

So add this to the steps list:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
.. literalinclude:: github-actions-ci-cd-sample/publish-to-testpypi.yml
:language: yaml
:start-after: version: 3.7
:end-before: Actualy publish to PyPIs
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
:end-before: Actualy publish to PyPIs
:end-before: Actually publish to PyPI/TestPyPI


Publishing dist to Test PyPI and production PyPI
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
Publishing dist to Test PyPI and production PyPI
Publishing the distribution to PyPI and TestPyPI
================================================

Finally, add the following steps at the end:

.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml
:language: yaml
:start-after: Actualy publish to PyPIs
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
:start-after: Actualy publish to PyPIs
:start-after: Actually publish to PyPI/TestPyPI

These two steps use the `pypa/gh-action-pypi-publish`_ GitHub
Action: the first one uploads contents of the ``dist/`` folder
into Test PyPI unconditionally and the second does that to
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
into Test PyPI unconditionally and the second does that to
into TestPyPI unconditionally and the second does that to
production PyPI but only if the current commit is tagged.
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
production PyPI but only if the current commit is tagged.
PyPI, but only if the current commit is tagged.


That's all, folks!
==================

Now, whenever you push a tagged commit to your Git repo remote
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
Now, whenever you push a tagged commit to your Git repo remote
Now, whenever you push a tagged commit to your Git repository remote
on GitHub, this workflow will publish it to PyPI.
And it'll publish any push to Test PyPI which is useful for
This conversation was marked as resolved by webknjaz

This comment has been minimized.

Copy link
@di

di Sep 19, 2019

Member
Suggested change
And it'll publish any push to Test PyPI which is useful for
And it'll publish any push to TestPyPI which is useful for
providing test builds to your alpha users as well as making
sure that your release pipeline remains healthy!


.. _API token: https://pypi.org/help/#apitoken
.. _GitHub Actions CI/CD: https://github.com/features/actions
.. _join the waitlist: https://github.com/features/actions/signup
.. _pypa/gh-action-pypi-publish:
.. _Secrets:
https://help.github.com/en/articles/virtual-environments-for-github-actions#creating-and-using-secrets-encrypted-variables
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.