Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Clone this wiki locally
Wheels are the new standard binary installation format for Python.
Many projects provide wheels now : http://pythonwheels.com
However, only some of these wheels are binary wheels.
Binary wheels on OSX
Some popular projects that already have such wheels are:
Question: will pip give me a broken wheel?
"Matching binary wheels" are wheels with a filename that matches the installing pip's Python version, Python ABI version, and the platform tag.
At first, some worried that this tag system might cause trouble for people installing OSX binary wheels into unusual Python versions, such as homebrew and macports.
In practice though, the platform tag does a good job of preventing pip from installing a wheel that won't work.
Here's an example wheel filename:
numpy-1.8.0-cp27-none-macosx_10_6_intel.whl. The name splits at dashes
- numpy (package name)
- 1.8.0 (package version)
- cp27 (CPython 2.7 - Python version)
- none (Python ABI number - only applies to Pythons >= 3)
- macosx_10_6_intel (platform tag)
The platform tag comes from the output of
python -c "import distutils.util; print(distutils.util.get_platform())" on the platform building the wheel.
I (MB) built
numpy-1.8.0-cp27-none-macosx_10_6_intel.whl with Python 2.7 from a
Python.org binary installer on an OSX 10.9 machine. The OSX platform tag
further breaks down to:
- 10_6 (the version of the SDK used to compile Python)
- intel (short-hand for a fat binary containing x86_64 and i386 objects)
As you see, the SDK part of the tag is not from the OSX running on the machine
I (MB) was building from but from the distutils configuration on the Python I
was building for. In this case I was building for a Python.org binary, and
that Python.org binary gave
macosx-10.6-intel from distutils
Up until pip 6.0, for pip to accept the wheel as matching its own platform,
these values have to match exactly. Meaning, that the Python running pip on
the installing machine has to have a value from
distutils.util.get_platform() that matches
(after converting dots and dashes to underscores).
Here are some values of
distutils.util.get_platform() for different Pythons on OSX:
|Python source||Python version||OSX version||
You get the idea. Python.org Pythons all use the 10.6 SDK, and have fat (x86_64 and i386) architecture in them. System Pythons use the SDK for the OSX they ship with, and also have fat architecture. Homebrew and Macports Python have the SDK for the OSX they are installing on, and x86_64 architecture only.
This tells us two things. First - wheels built with Python.org Python will
have platform tags that only match other Python.org Python installations, for pip < 6.0.
So, for pip < 6.0, pip won't install them into system Python (wrong SDK) or
homebrew or macports Python (wrong SDK and different architecture). Second,
the Python.org wheels will in fact have the correct architecture and
compatible SDK for all the other Pythons listed. Why? Because having a fat
binary includes having x86_64, so is compatible with x86_64-only builds.
Stuff compiled with the 10.6 SDK should also be compatible with stuff built
against later SDK versions (up to and including 10.9). You can demonstrate
this to yourself by renaming the wheel above to - for example -
numpy-1.8.0-cp27-none-macosx_10_9_x86_64.whl and then installing into a
homebrew python on OSX 10.9. Sure enough, it installs, imports and tests
Answer: no, pip will be very careful to give you a matching wheel
Python.org wheels are safe to distribute because:
- pip < 6.0 will only install the wheel into a Python.org Python by default and
- The architecture and SDK versions are in fact compatible with system Python, homebrew Python and macports Python.
In fact, because it is possible to work out which wheels are compatible between Pythons, Min Ragan-Kelley proposed that pip should recognize that prior SDKs match later OSX versions, and fat binaries should match sensible single architectures (PR and discussion). This change was merged into the pip master branch on June 12 2014 and was released as part of pip 6.0.
So - nothing can go wrong with OSX binary wheels?
Strange to say, things can go wrong with wheels as for any binary distribution. Here are some things that can go wrong, and how to fix them:
Linking to external libraries that some machines do not have
All Python extensions link against OSX system libraries, but these are carefully managed to be ABI compatible between OSX versions, and you should not run into problems with these.
This library is present and ABI compatible for all of OSX versions 10.6 and higher.
If you build a complicated Python extension it may link against some external
libraries elsewhere on the system. scipy is
one example; it links to the gfortran runtime libraries, whereever it finds
them. Here's the output of
delocate-listdeps --all for a scipy wheel
built naively on a standard OSX 10.9 system using gfortran from homebrew:
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate /usr/lib/libSystem.B.dylib /usr/lib/libstdc++.6.dylib /usr/local/Cellar/gfortran/4.8.2/gfortran/lib/libgcc_s.1.dylib /usr/local/Cellar/gfortran/4.8.2/gfortran/lib/libgfortran.3.dylib /usr/local/Cellar/gfortran/4.8.2/gfortran/lib/libquadmath.0.dylib
Again, the libraries in
/usr/lib will be present on OSX >=
10.6, but of course the libraries in
/usr/local/Cellar/gfortran will only be
present if someone has installed gfortran via homebrew. If I distribute this
wheel, it will only work for someone who has installed these libraries. The
delocate utility can usually fix this by copying the dynamic libraries into
the wheel and relinking the extensions.
Other stuff that might happen but we haven't yet seen
Some people have reported that binaries built with an earlier SDK (such as 10.3) on a later OSX OS (such as 10.9) do not in fact work on earlier versions of OSX, as they should (see comment on pip PR). I have not myself (MB) run into this problem with the 10.6 SDK. For safety, it is best to build binaries such as wheels on the same OSX versions as the SDK. For example, if you are building wheels targeting the 10.6 SDK, try and build the wheels on a machine running 10.6. I don't know of any reports of problems using these binaries on later OSX versions.
There was some worry on a pip pull-request discussion that it might be possible to get Python confused with wheels built against different C++ runtime libraries. Min RK couldn't make this problem happen with test-cases, so we are currently working on the assumption that this is not an issue. Obviously it doesn't come up if you're not using C++.
The answer is always the same - test
NIPY_URL=https://nipy.bic.berkeley.edu/scipy_installers pip install --find-links $NIPY_URL tornado
Do this in virtualenvs with different Pythons and on different OSX versions. If you run into trouble, let us know via the Python Mac special interest group mailing list and we'll try to help. At very least, we'd really like to know.
Practical example of building wheels for a project
As we've seen, the MacPython Python distributions are the best to build against, because they use the 10.6 SDK (and hence are compatible with OSX versions from 10.6) and they have dual architectures (i386 and x86_64). This makes the resulting wheel compatible with system Python, homebrew and macports.
- MacPython 2.7
- MacPython 3.3
- MacPython 3.4
For MacPython 2.7 and 3.3 you'll need an up-to-date pip (3.4 comes with pip):
export MACPIES=/Library/Frameworks/Python.framework/Versions $MACPIES/2.7/bin/python get-pip.py $MACPIES/3.3/bin/python3 get-pip.py
See instructions at http://pip.readthedocs.org/en/latest/installing.html
$MACPIES/2.7/bin/pip install wheel $MACPIES/3.3/bin/pip3 install wheel $MACPIES/3.4/bin/pip3 install wheel
Here I'm building wheels for
cd markupsafe rm -rf build $MACPIES/2.7/bin/python setup.py bdist_wheel rm -rf build $MACPIES/3.3/bin/python3 setup.py bdist_wheel rm -rf build $MACPIES/3.4/bin/python3 setup.py bdist_wheel
You should now have three wheels in your distribution directory (usually
dist). In my case:
dist/MarkupSafe-0.23-cp27-none-macosx_10_6_intel.whl dist/MarkupSafe-0.23-cp33-cp33m-macosx_10_6_intel.whl dist/MarkupSafe-0.23-cp34-cp34m-macosx_10_6_intel.whl
Check wheels for external dependencies that need shipping
pip install delocate delocate-listdeps dist/*.whl
Markupsafe wheels have no dependencies outside the system library paths, so you get something like this:
dist/MarkupSafe-0.23-cp27-none-macosx_10_6_intel.whl: dist/MarkupSafe-0.23-cp33-cp33m-macosx_10_6_intel.whl: dist/MarkupSafe-0.23-cp34-cp34m-macosx_10_6_intel.whl:
If your project does have some dependencies from the analysis above, then:
mkdir fixed_wheels delocate-wheel -w fixed_wheels dist/*.whl
Finally, if you want to make the wheel installable on any platform by pip < 6.0,
you might want to rename the wheels to express the fact that they will
work on system Python and homebrew / macports for 10.9. For example, you might
There's a script to do that in this gist
This renaming is no longer required with pip >= 6.0.
Upload to pypi
Automating wheel builds with travis
We (the MacPython organization) support some other projects building OSX wheels using travis-ci.org; feel free to contact us if you'd like help too.
See Wheel building for details.