Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip wheel does not cache wheels it built locally #6852

Closed
sbidoul opened this issue Aug 10, 2019 · 0 comments · Fixed by #7285
Closed

pip wheel does not cache wheels it built locally #6852

sbidoul opened this issue Aug 10, 2019 · 0 comments · Fixed by #7285

Comments

@sbidoul
Copy link
Member

@sbidoul sbidoul commented Aug 10, 2019

Environment

  • pip version: 19.2
  • Python version: 3.7 (probably irrelevant)
  • OS: Linux (probably irrelevant, but necesary for the reproducer below)

Description

pip wheel does not cache wheels it built locally, whereas pip install does.

My use case for pip wheel is mostly pip wheel -r frozen-requirements.txt to build
and collect everything needed for testing on CI and deployment. For wheels that have a long build time it would help to have them cached too in such a scenario.

Expected behavior

pip wheel should cache all wheels it built locally (i.e. built from sdists on pypi),
like pip install does.

How to Reproduce

Here is an example with pyyaml 5.1.2 wich, as of today, has no linux wheel on PyPI.

First make sure the pip cache is empty.

Use pip wheel, notice it uses the (possibly cached) sdist, and builds the wheel.
Repeat and notice the behaviour remains the same.

$ venv/bin/pip wheel pyyaml==5.1.2
Collecting pyyaml==5.1.2
  Using cached https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz
Building wheels for collected packages: pyyaml
  Building wheel for pyyaml (setup.py) ... done
  Created wheel for pyyaml: filename=PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl size=44104 sha256=0c0a590c4188291ddce3d984cab5d4868115181183a3972fdf95560cc9530f93
  Stored in directory: /tmp/brol1
Successfully built pyyaml

Next, pip install the same, and notice pip builds and caches the wheel.

$ venv/bin/pip install pyyaml==5.1.2
...
  Building wheel for pyyaml (setup.py) ... done
  Created wheel for pyyaml: filename=PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl size=44104 sha256=a8625826f5bdcb1434d9d8339542db1d47d3f84bf27c2180c39345f34d98821c
  Stored in directory: ~/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030
...

Finally pip wheel again and notice it uses the cached wheel.

$ rm PyYAML-5.1.2*
$ venv/bin/pip wheel pyyaml==5.1.2
Collecting pyyaml==5.1.2
  Saved ./PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl
Skipping pyyaml, due to already being wheel.

Analysis

I noticed this issue while working on #6640.

I think the problem arises in this area.
The output directory provided to _build_one is either the cache (when autobuilding), or the target directory provided by the user in case of pip wheel, in which case the cache is bypassed.

A solution could be to always build to the cache directory, and in case we are not autobuilding copy wheel_file to self._wheel_dir after building.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

2 participants
You can’t perform that action at this time.