Skip to content
This repository has been archived by the owner on Feb 15, 2023. It is now read-only.

Commit

Permalink
DOC: update README
Browse files Browse the repository at this point in the history
[skip ci]
  • Loading branch information
matthew-brett committed Feb 7, 2018
1 parent b9f1e84 commit cce21a2
Showing 1 changed file with 70 additions and 43 deletions.
113 changes: 70 additions & 43 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,75 +1,84 @@
####################################
Building and uploading Cython wheels
####################################
########################################
Building and uploading kiwisolver wheels
########################################

We automate wheel building using this custom github repository that builds on
the travis-ci OSX machines, travis-ci Linux machines, and the Appveyor VMs.
the travis-ci OSX machines and the travis-ci Linux machines.

The travis-ci interface for the builds is
https://travis-ci.org/MacPython/cython-wheels
https://travis-ci.org/MacPython/kiwisolver-wheels

Appveyor interface at
https://ci.appveyor.com/project/matthew-brett/cython-wheels
https://ci.appveyor.com/project/matthew-brett/kiwisolver-wheels

The driving github repository is
https://github.com/MacPython/cython-wheels
https://github.com/MacPython/kiwisolver-wheels

Quickstart
==========
Using the repository
====================

For building Cython version 0.26.2::
The repository contains the branches:

python trigger_build.py 0.26.2
* ``master`` - for development and daily builds;
* ``vx.y.z`` - for building releases.

Then wait for the builds to finish. Check:
Travis-CI and Appveyor builds the ``master`` regularly (daily/weekly),
via `Travis-CI cron jobs
<https://docs.travis-ci.com/user/cron-jobs/>`_ and `Appveyor scheduled
builds
<https://www.appveyor.com/docs/build-configuration/#scheduled-builds>`.

* https://travis-ci.org/MacPython/cython-wheels
* https://ci.appveyor.com/project/matthew-brett/cython-wheels
* The Rackspace container where the wheels go:
https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com
Builds from the ``master`` branch upload to a Rackspace container for
pre-releases at
https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com

When they are all done::
Builds from the release branches upload to a Rackspace container for releases
at
https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com

pip install beautifulsoup4 twine
python upload_wheels.py 0.26.2
Pull requests should usually be submitted to the ``master`` branch.

How it works
============

The wheel-building repository:

* does a fresh build of any required C / C++ libraries;
* builds a Cython wheel, linking against these fresh builds;
* builds a kiwisolver wheel, linking against these fresh builds;
* processes the wheel using delocate_ (OSX) or auditwheel_ ``repair``
(Manylinux1_). ``delocate`` and ``auditwheel`` copy the required dynamic
libraries into the wheel and relinks the extension modules against the
copied libraries;
* uploads the built wheels to http://wheels.scipy.org (a Rackspace container
kindly donated by Rackspace to scikit-learn).
* uploads the built wheels to a Rackspace container - see "Using the
repository" above. The containers were kindly donated by Rackspace to
scikit-learn).

The resulting wheels are therefore self-contained and do not need any external
dynamic libraries apart from those provided as standard by OSX / Linux as
defined by the manylinux1 standard.

The ``.travis.yml`` file in this repository has a line containing the API key
for the Rackspace container encrypted with an RSA key that is unique to the
repository - see http://docs.travis-ci.com/user/encryption-keys. This
repository - see https://docs.travis-ci.com/user/encryption-keys. This
encrypted key gives the travis build permission to upload to the Rackspace
directory pointed to by http://wheels.scipy.org.
containers we use to house the uploads.

Triggering a build
==================

You will likely want to edit the ``.travis.yml`` and ``appveyor.yml`` files to
specify the ``BUILD_COMMIT`` before triggering a build - see below.

For releases, use an existing release branch, or push a new release
branch to the repository.

You will need write permission to the github repository to trigger new builds
on the travis-ci interface. Contact us on the mailing list if you need this.

You can trigger a build by:

* making a commit to the ``cython-wheels`` repository (e.g. with ``git commit
--allow-empty``); or
* making a commit to the `kiwisolver-wheels` repository (e.g. with `git
commit --allow-empty`); or
* clicking on the circular arrow icon towards the top right of the travis-ci
page, to rerun the previous build.

Expand All @@ -78,24 +87,31 @@ a new set of build products and logs, keeping the old ones for reference.
Keeping the old build logs helps us keep track of previous problems and
successful builds.

Which cython commit does the repository build?
Which kiwisolver commit does the repository build?
===============================================

The ``cython-wheels`` repository will build the commit specified in the
``BUILD_COMMIT`` at the top of the ``.travis.yml`` and ``appveyor.yml`` files.
This can be any naming of a commit, including branch name, tag name or commit
hash.
The `kiwisolver-wheels` repository will build the commit specified in the
``BUILD_COMMIT`` at the top of the ``.travis.yml`` file and ``appveyor.yml``
files. This can be any naming of a commit, including branch name, tag name or
commit hash.

Note: when making a kiwisolver release, it's best to only push the commit (not the
tag) of the release to the ``kiwisolver`` repo, then change ``BUILD_COMMIT`` to the
commit hash, and only after all wheel builds completed successfully push the
release tag to the repo. This avoids having to move or delete the tag in case
of an unexpected build/test issue.

Uploading the built wheels to pypi
==================================

Be careful, http://wheels.scipy.org points to a container on a distributed
content delivery network. It can take up to 15 minutes for the new wheel file
to get updated into the container at http://wheels.scipy.org.
* pre-releases container visible at
https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com
* release container visible at
https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com

The same contents appear at
https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com;
you might prefer this address because it is https.
Be careful, these links point to containers on a distributed content delivery
network. It can take up to 15 minutes for the new wheel file to get updated
into the containers at the links above.

When the wheels are updated, you can download them to your machine manually,
and then upload them manually to pypi, or by using twine_. You can also use a
Expand All @@ -109,9 +125,9 @@ You will typically have a directory on your machine where you store wheels,
called a `wheelhouse`. The typical call for `wheel-uploader` would then
be something like::

VERSION=0.24.2
VERSION=1.0.1
CDN_URL=https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com
wheel-uploader -u $CDN_URL -s -v -w ~/wheelhouse -t all Cython $VERSION
wheel-uploader -u $CDN_URL -s -v -w ~/wheelhouse -t all kiwisolver $VERSION

where:

Expand All @@ -122,11 +138,22 @@ where:
* ``-w ~/wheelhouse`` means download the wheels from to the local directory
``~/wheelhouse``.

``cython`` is the root name of the wheel(s) to download / upload, and
``0.24.2`` is the version to download / upload.
``kiwisolver`` is the root name of the wheel(s) to download / upload, and
``1.0.1`` is the version to download / upload.

In order to upload the wheels, you will need something like this
in your ``~/.pypirc`` file::

[distutils]
index-servers =
pypi

[pypi]
username:your_user_name
password:your_password

So, in this case, ``wheel-uploader`` will download all wheels starting with
``Cython-0.24.2-`` from http://wheels.scipy.org to ``~/wheelhouse``, then
So, in this case, `wheel-uploader` will download all wheels starting with
`kiwisolver-1.0.1-` from the URL in ``$CDN_URL`` above to ``~/wheelhouse``, then
upload them to PyPI.

Of course, you will need permissions to upload to PyPI, for this to work.
Expand Down

0 comments on commit cce21a2

Please sign in to comment.