python wheel

[gjcarneiro]

Would it be possible, from now on, to also generate and upload Python wheels? It will speed up installation of this package a lot...

[brian-brazil]

This package is pure python, would it make a difference?

[gjcarneiro]

Yes, it makes a huge difference, installation from Python wheel, even for pure Python, is so much faster, as installation is little more than downloading a zip file and uncompressing it directly to the final destination. A normal package installation, not only needs uncompressing to a temporary src directory, then it needs to execute python setup.py install, which is a lot slower.

https://pythonwheels.com

[brian-brazil]

Can you quantify "a lot"? 200kB doesn't take a lot to process.

[xavfernandez]

It mainly saves pip from spawning a process to launch python setup.py bdist_wheel and automagically create the wheel :)

(tmp-f8893c0a100797) $ pip uninstall prometheus-client -y
Uninstalling prometheus-client-0.7.1:
 Successfully uninstalled prometheus-client-0.7.1
(tmp-f8893c0a100797) $ time pip install --no-cache-dir --no-index ~/Downloads/prometheus_client-0.7.1.tar.gz
Processing /home/xfernandez/Downloads/prometheus_client-0.7.1.tar.gz
Building wheels for collected packages: prometheus-client
 Building wheel for prometheus-client (setup.py) ... done
 Created wheel for prometheus-client: filename=prometheus_client-0.7.1-cp37-none-any.whl size=41402 sha256=66668319af083af049349c7e8e81f7def84260e36c9e0968b753e27fefd5c421
 Stored in directory: /tmp/pip-ephem-wheel-cache-sh8aaupe/wheels/c3/4f/4e/65ac82a9f5c3008dee932dbf792e6cf5f9a3ae9de9031735a9
Successfully built prometheus-client
Installing collected packages: prometheus-client
Successfully installed prometheus-client-0.7.1

real    0m0.760s
user    0m0.677s
sys     0m0.083s
(tmp-f8893c0a100797) $ pip uninstall prometheus-client -y
Uninstalling prometheus-client-0.7.1:
 Successfully uninstalled prometheus-client-0.7.1
(tmp-f8893c0a100797) [xfernandez@xaferiba tmp-f8893c0a100797]$ time pip install --no-cache-dir --no-index ~/Downloads/prometheus_client-0.7.1-py3-none-any.whl 
Processing /home/xfernandez/Downloads/prometheus_client-0.7.1-py3-none-any.whl
Installing collected packages: prometheus-client
Successfully installed prometheus-client-0.7.1

real    0m0.370s
user    0m0.321s
sys     0m0.047s

(the wheel was produced via pip wheel ~/Downloads/prometheus_client-0.7.1.tar.gz -w ~/Downloads/)

[xavfernandez]

Since pip will automatically build the wheel at the first install, it would save quite a lot of computation (~300 ms multiplied by every fresh install) to build the wheel once during the release and upload it to PyPI.org.

This would mean switching from:

git checkout <tag>
pip install -U setuptools twine
python setup.py sdist
twine upload dist/prometheus_client-<version>.tar.gz

to

git checkout <tag>
pip install -U setuptools twine wheel
python setup.py sdist bdist_wheel --universal
twine upload dist/prometheus_client-<version>.tar.gz dist/prometheus_client-<version>-py2.py3-none-any.whl

[brian-brazil]

I'm not sure complicating our build process is worth saving 300ms of install cost.

[xavfernandez]

I'm not sure complicating our build process is worth saving 300ms of install cost.

It's 300 ms multiplied by the 100k+ installs of prometheus-client that happen everyday according to https://pypistats.org/packages/prometheus-client.
So providing a pure-python wheel could potentially save several hours of computation everyday :)

If you could point me to your build process, I'd be more than pleased to improve it.

[brian-brazil]

I build and release by hand.

[gjcarneiro]

Indeed, it's usually trivial to build/upload a wheel: if now you are doing python setup.py sdist upload, just change to python setup.py sdist bdist_wheel upload. You may also need to create a setup.cfg file containing:

[bdist_wheel]
universal = 1

This tells it that the built wheel is pure Python and works in either Python 2 or Python 3 without changes.

[xavfernandez]

I've opened #487 if you don't like the additional setup.cfg.

As @gjcarneiro explained, if you're currently using python setup.py sdist upload you'd just have to switch to python setup.py sdist bdist_wheel upload (either with a setup.cfg file or with #487 merged).