Upload wheels to PyPI

[johnthagen]

Wheels (.whl) for this package are currently missing from PyPI. Could wheels be uploaded for the current and future releases?

Read more about the advantages of wheels to understand why generating wheel distributions are important.

To create a wheel along with source distribution:

(venv) $ pip install --upgrade pip setuptools wheel
(venv) $ python setup.py sdist bdist_wheel

# See dist/*.whl

To upload wheels:

(venv) $ pip install twine
(venv) $ twine upload dist/*

[bsolomon1124]

Wheels are great of course, but this project contains C extensions, so the full set of wheels would not be doable through one bdist_wheel call on any arbitrary host platform.

[nsoranzo]

@bsolomon1124 It seems there is an initial implementation of wheel building using cibuildwheel in .travis.yml , but only under macOS and the built wheels are not uploaded to PyPI.

[bjmc]

I'd like to chip and second this request, if it's possible. We're running into a problem deploying a Chalice application that depends on pyrsistent from a MacOS workstation. The issue is the same one described here and the suggestion is that if PyPI included prebuilt manylinux1 wheels, then we wouldn't need to compile the dependency as part of the deploy process.

Thanks for all your work on this project!

[odidev]

As per above discussion, we know that pyrsistent doesn't have wheel for aarch64/x86 on PyPI repository. So, while installing pyrsistent via pip on aarch64/x86 machine, pip builds the source code resulting in it takes more time to install pyrsistent. Making wheel available for aarch64/x86 will benefit aarch64/x86 users by minimizing pyrsistent installation time. We can release the wheel using CI/CD tools such as Travis-ci, circle-ci, azure-pipeline etc...

@tobgu, please let me know if I can help you building wheel/uploading to PyPI repository. I am curious to make pyrsistent wheel available for aarch64. It will be a great opportunity for me to work with you.

[tobgu]

Hi, thanks for reporting this and sorry for not taking the time to respond earlier. I realise that adding wheels is something that many of you would like (see also the older issue #88) but I haven't really gotten to it.

I imagine what I would like to do is from Travis, or similar, build and push a number of wheels on successful build of a tagged release. There may be better ways of doing it though... One thing I know I don't want for sure is to maintain a bunch of VMs for the different targets and build the wheels myself.

So, I'm now asking for some input to be able to move forward on this:

1. Which platforms/targets are of interest?
2. Could you provide me with examples of other projects doing something similar in a good way that I can learn from?

Thanks!

[kgaughan]

@tobgu I think the main platform the people are interested in is Linux, specifically because pyrsistent gets used used in AWS Lambda via Chalice, and thus manylinux1_x86_64 is probably what you should be targetting first, followed by manylinux1_aarch64. See PEP-0425 for information on Wheel tags.

The two projects I know of that are probably worth looking at for inspiration are cryptography and cffi. I haven't dove too deeply into how they're building their wheels, but from what I can see, cffi might be an easier starting point.

[johnthagen]

It can also help in some Docker contexts, as some times you can generate slimmer images (also simpler to maintain) if you can avoid having to include a C compiler.

[kgaughan]

@tobgu While looking up another issue, I stumbled across this, which may be useful to you: https://github.com/joerick/cibuildwheel

It looks solid, the documentation is good, and it's used by some prominent projects that should work as good examples to draw from.

[johnthagen]

The manyliunx project has pre-configured Docker images to help building manylinux Docker wheels.

opencv-python uses them to build manylinux2014 wheels for Linux.

[JanneKiiskila]

Or should the requirements.txt just ask for wheels, since it clearly needs it? @kgaughan

• Stumbled into this problem today myself, our tool installation fails due to pyrsistent really needing wheels, but it's not asking for those.

[kgaughan]

@JanneKiiskila No, that wouldn't work. This is, unfortunately, a required dependency of Chalice, and if try to tell pip just to use wheels (which is what you want to do if you just want wheels), it'll fail. That's what's breaking for me. For this to work, there would need to be a wheel built for pyrsistent, and there is none. And I need to force the use of wheels as I'm deploying the lambda from Mac OS, and pip is disappointingly dumb when it comes handling cross-platform deployments. I very much which I could just tell it to use source distributions where necessary but that it's target should be manylinux and to die if it actually need to build any binaries.

[tobgu]

Thanks @johnthagen for contributing to this happening. I've released wheels for 0.17.4rc1 now. It would be great with some testing and feedback before I cut a proper release if anyone has the chance to do so.

[johnthagen]

@tobgu I tested install on Windows x64, Python 3.9 and it seems to install correctly:

> python -m pip install pyrsistent==0.17.4rc1
Collecting pyrsistent==0.17.4rc1
  Downloading pyrsistent-0.17.4rc1-cp39-cp39-win_amd64.whl (62 kB)
     |████████████████████████████████| 62 kB 1.5 MB/s
Installing collected packages: pyrsistent
Successfully installed pyrsistent-0.17.4rc1
(venv3) PS C:\Users\User\temp> python
Python 3.9.4 (tags/v3.9.4:1f2e308, Apr  6 2021, 13:40:21) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyrsistent
>>>

> python -m pip list
Package    Version
---------- ---------
pip        21.1.2
pyrsistent 0.17.4rc1
setuptools 57.0.0

[johnthagen]

A Linux Docker install also worked.

FROM python:3.8-slim-buster

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONBUFFERED 1
ENV PIP_NO_CACHE_DIR 1
ENV WORKDIR /src

WORKDIR ${WORKDIR}

RUN pip install --upgrade pip wheel && \
    pip install pyrsistent==0.17.4rc1

ENTRYPOINT ["python"]

> docker build --progress=plain --no-cache --tag image .
#1 [internal] load build definition from Dockerfile
#1 sha256:9821e2316c5caca8de82be6ec1d1df5d367f4a0bdac3a886af0cd58ce1b7408e
#1 transferring dockerfile: 32B done
#1 DONE 0.1s

#2 [internal] load .dockerignore
#2 sha256:ad5cd11e6950e44fa392a08e1b24e93c646ad1c2ee9cbdcb624e09d944049b9d
#2 transferring context: 35B done
#2 DONE 0.1s

#3 [internal] load metadata for docker.io/library/python:3.8-slim-buster
#3 sha256:a82ddfb0a3c3ab3f4e2ebc7582cec39f26df7d1ae41d54f70ea9fe596d7b25c7
#3 DONE 0.0s

#6 [1/3] FROM docker.io/library/python:3.8-slim-buster
#6 sha256:88461ab56661347448ec72a8bb2b6302d3f317a1021310b6e70782cccc8113b7
#6 DONE 0.0s

#4 [2/3] WORKDIR /src
#4 sha256:47389e9323663eb8b3272c8d5d0ef1eec3f0d35afe90595569cf48f15aa861a6
#4 CACHED

#5 [3/3] RUN pip install --upgrade pip wheel &&     pip install pyrsistent==0.17.4rc1
#5 sha256:5c9b511f0f82a61a9188bf067d41e485b2a3270f478ceebae1891b02430ac9e9
#5 1.177 Requirement already satisfied: pip in /usr/local/lib/python3.8/site-packages (20.3.3)
#5 2.291 Requirement already satisfied: wheel in /usr/local/lib/python3.8/site-packages (0.36.2)
#5 2.940 Collecting pip
#5 3.195   Downloading pip-21.1.2-py3-none-any.whl (1.5 MB)
#5 3.480 Installing collected packages: pip
#5 3.481   Attempting uninstall: pip
#5 3.481     Found existing installation: pip 20.3.3
#5 3.592     Uninstalling pip-20.3.3:
#5 3.700       Successfully uninstalled pip-20.3.3
#5 4.423 Successfully installed pip-21.1.2
#5 5.464 Collecting pyrsistent==0.17.4rc1
#5 5.629   Downloading pyrsistent-0.17.4rc1-cp38-cp38-manylinux1_x86_64.whl (118 kB)
#5 5.717 Installing collected packages: pyrsistent
#5 5.770 Successfully installed pyrsistent-0.17.4rc1
#5 5.770 WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
#5 DONE 6.1s

#7 exporting to image
#7 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#7 exporting layers
#7 exporting layers 0.2s done
#7 writing image sha256:6deefc5d8024c44641e7c1a214bebeb6d608878b7451719480214be36e76620a 0.0s done
#7 naming to docker.io/library/fact done
#7 DONE 0.3s

> docker run --rm --interactive --tty image
Python 3.8.7 (default, Jan 12 2021, 17:16:32)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyrsistent
>>>