Wheel building

Peter Cock edited this page Jul 18, 2017 · 2 revisions

MacPython OSX wheel building

The MacPython organization houses some repositories that projects use for building OSX, Linux and Windows wheels.

How the build repos work

The repos work via continuous integration services travis-ci (covering Mac and Linux) and appveyor (for Windows).

Triggering a build via the travis-ci web interface or a commit to the repo causes the .travis.yml file to be run, executing these steps:

  • download and install of Python.org Python (see Spinning Wheels)
  • build of a standard OSX wheel (however the project defines 'standard')
  • wheel install into a virtualenv
  • tests using the project's standard testing procedure
  • upload of built wheel to a Rackspace container pointed to by http://wheels.scikit-image.org

Likewise building the Windows wheels can be triggered via the appveyor website or a commit to the repository, which causes the appveyor.yml file to be executed on a virtual machine.

The upload step

In order for the upload to work, the .travis.yml needs to have a section like this:

before_deploy: cd matplotlib/dist
deploy:
provider: cloudfiles
username: travis-worker
api_key:
    secure: K9JEFezfB40JDYntdoBKfxFD6qAlWGo1WaNM269JUAS6+8taD8gLOe83r1hyZWa0m2hdTilk0ySDUxjs+aTe2f4+ES+s8aiORSTFHChMUL/7+vteaHUle3BEuUDx23mnx5XNQT/n1gX2WkWwqZMH7JDAsHbexuquQZfJVAWRpo4=
region: ORD
container: wheels
skip_cleanup: true

The api_key secure entry is a Rackspace API key encrypted against the travis encryption keys of the repository. See the link for details, but in short, to set these up, you probably want to install the travis command line client, cd into the wheel building repo, and run:

travis encrypt <the rackspace API key>

where "<the rackspace API key>" is the key referred to below. This gives you output something like:

secure: "fC8a017zdgsbkxZRcJrIrEa35LyzJJEyxTHkbMG/gVJKLhzMLKdw7F6FnsUyxu/RHA5FHE5D2S7X471dNU/xfCYJjWkzvHlrZrwQ5vwJ0KGIRE46KiO0URBGoGFQWI1WPhOeirIJ0ZLaahTYLbXYnQRhM2mOjCmIY74jPSTrrvg="

which you can put back into the .travis.yml file, replacing the current api_key section.

Similarly for your appveyor.yml file you also need to provide an API key under environment:, global:, as in the scikit-learn-wheels configuration:

environment:
  global:
    WHEELHOUSE_UPLOADER_USERNAME: sklearn-appveyor
    WHEELHOUSE_UPLOADER_SECRET:
      secure: P1/NpWGueg+26XlcS0P6kIKO1KbRw67S6ltwEjxb+vgf0F+zfNT6FRaMPKfHSXiR

Upload to pypi

When the wheels are available in the Rackspace container, you can upload these using the wheel-uploader script in the terryfy repo.

The Rackspace account

The Rackspace account belongs to the scikit-learn team. Rackspace donated these resources to the scikit-learn team, and Olivier Grisel kindly let us use them.

The wheel builders use a sub-account that can upload to a Rackspace cloud files container (see Rackspace cloud files) on a content delivery network (CDN) (see Rackspace CDN).

The scikit-image team set up a URL to point to the container at http://wheels.scikit-image.org. You can also use the raw rackspace URL.

Because the container is on a content delivery network (CDN), the files do not become live until a short while after they get uploaded. New files appear relatively quickly (say 5 minutes). You may have to wait up to 15 minutes for files to update from previous versions. This is due to the CDN time-to-live feature. If you are waiting for new wheels to arrive, you may want to wait 15 minutes, or refresh the directory listing to get the file modification times.

Chris Barker and Matthew Brett and Olivier Grisel have the credentials to this account at the moment. If you need us to set up the travis key (above), please open an issue on a wheel building repository and let us know via email or by using the github @matthew-brett etc notification feature.

See Managing Rackspace.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.