Skip to content

Add auto-publishing to PyPI #487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 18, 2020
Merged

Add auto-publishing to PyPI #487

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,15 @@ script:
after_success:
- if [[ $TRAVIS_PYTHON_VERSION != pypy* ]]; then pip install coveralls --use-mirrors ; coveralls ; fi

# publish new version to pypi
deploy:
provider: pypi
Copy link

@bsolomon1124 bsolomon1124 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If desired, you could test this first with Test PyPI:

https://docs.travis-ci.com/user/deployment/pypi/#releasing-to-a-self-hosted-pypi

E.g. I think that would be:

server: https://test.pypi.org/simple/

skip_existing: true
on:
tags: true
Copy link

@bsolomon1124 bsolomon1124 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on: tags: true will publish from any tag on any branch, no?

Maybe it would be more desirable to only publish when something on master is tagged? https://docs.travis-ci.com/user/deployment#conditional-releases-with-on. Also, not sure if possible, but Travis may offer a way to match the tag to only a final release (e.g. 1.2.3, not 1.2.3.dev0) as defined by PEP 440.

Copy link
Contributor Author

@filips123 filips123 May 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on: tags: true will publish from any tag on any branch, no?

Yes it will publish on any tag on any branch. But because tags are almost always published on master, it shouldn't cause much problems. And even it is would trigger on multiple branches for same tag, it won't deploy it multiple times because of skip_existing: true.

I think that Travis CI allows conditional deployments to be either on all tags on all branches or commits on specific branch, so publishing only on tags on master could be harder. It may be possible with custom condition but I'm now sure how exactly will this work.

Also, not sure if possible, but Travis may offer a way to match the tag to only a final release (e.g. 1.2.3, not 1.2.3.dev0) as defined by PEP 440.

It's probably also possible with custom condition. But I also don't know if this is really needed because pip and PyPI will handle pre-releases and not install them unless specifically required by user.

Copy link

@bsolomon1124 bsolomon1124 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that Travis CI allows conditional deployments to be either on all tags on all branches or commits on specific branch, so publishing only on tags on master could be harder. It may be possible with custom condition but I'm now sure how exactly will this work.

So, I am not a Travis expert at all, but reading between the lines here,
it looks like this may be satisfied with:

  on:
    branch: staging
    tags: true

It's probably also possible with custom condition. But I also don't know if this is really needed because pip and PyPI will handle pre-releases and not install them unless specifically required by user.

For this, I was more getting at the possibility that a tag is pushed that doesn't represent a PEP-440 compliant version at all, such as git tag -a "this-is-not-a-version" ... git push ... --follow-tags. I agree with you that it is fine to upload development, pre-release, and post-release tags because PyPI very much can handle them as you point out.

Copy link
Contributor Author

@filips123 filips123 May 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this:

tags: true: deployment is triggered if and only if $TRAVIS_TAG is set. Depending on your workflow, you may set $TRAVIS_TAG explicitly, even if this is a non-tag build when it was initiated. This causes the branch condition to be ignored.

However, I don't know if this means that branch condition is ignored only if you manually set $TRAVIS_TAG or in any case when $TRAVIS_TAG is set.

user: $PYPI_USER
password: $PYPI_PASSWORD
Copy link

@bsolomon1124 bsolomon1124 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Travis docs mention "It is also possible, but not recommended, to use PyPI user and password, instead of token" -- though they don't actually mention the advantage of one over the other.

Copy link
Contributor Author

@filips123 filips123 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are more secure because:

  • Are longer so they probably take longer to brute force
  • And if they are hacked in some way, they are limited to specific project and can be revoked more easily

With this PR, it is possible to use password or token, because when using token you just set username to __token__ and password to actual token and this can be done with environment variables.

distributions: "sdist bdist_wheel"
Copy link

@bsolomon1124 bsolomon1124 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a bonus here, you could add a call to check-wheel-contents on the resulting wheel artifact - in this case it might look like:

check-wheel-contents --toplevel=toolz,tlz dist/toolz-*.whl

Copy link
Contributor Author

@filips123 filips123 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is harder to to because Travis will build source and wheel distributions and then immediately publish it to PyPI, without any place to run this check. But it may be possible to solve it with manually building source and wheels, checking wheels and publishing them using normal commands.

Copy link

@bsolomon1124 bsolomon1124 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, I did not consider that. I guess at best it would be a "post-upload" sanity check that you could remedy by fixing and releasing a subsequent patch version that addresses whatever issue was found in the wheel. The alternative would be to execute a job prior to deploy that builds the wheel and runs check-wheel-contents on that, then only deploy s if check-wheel-contents exits 0. But I concede that's kind of a hassle to build the wheel twice.

Copy link
Contributor Author

@filips123 filips123 May 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I though about adding check-wheel-contents to after_deploy section. However, "after_deploy does not affect the status of the build" so you will have to manually check build output.


notifications:
email: false