Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip 21.1 install with --no-binary fails while 21.0.1 succeeds (pep517) #9878

1 task done
g8trbryant opened this issue Apr 26, 2021 · 1 comment · Fixed by #9883
1 task done

pip 21.1 install with --no-binary fails while 21.0.1 succeeds (pep517) #9878

g8trbryant opened this issue Apr 26, 2021 · 1 comment · Fixed by #9883
type: bug A confirmed bug or unintended behavior


Copy link


Installing the pip package pex with --no-binary fails after pip version changed from 21.0.1 to 21.1. In the output log below, it seems like pep517 is involved. Steps below are successful if pip==21.0.1 is used.

Expected behavior

No response

pip version


Python version



RHEL7 x86_64

How to Reproduce

python3 -m venv venv
. venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade setuptools
python3 -m pip install pex==2.1.30 --no-binary :all:


Cache entry deserialization failed, entry ignored
Collecting pip
  Using cached
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
Successfully installed pip-21.1

Requirement already satisfied: setuptools in ./venv/lib/python3.6/site-packages (28.8.0)
Collecting setuptools
  Using cached setuptools-56.0.0-py3-none-any.whl (784 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 28.8.0
    Uninstalling setuptools-28.8.0:
      Successfully uninstalled setuptools-28.8.0
Successfully installed setuptools-56.0.0

Collecting pex==2.1.30
  Downloading pex-2.1.30.tar.gz (3.3 MB)
     |████████████████████████████████| 3.3 MB 3.0 MB/s 
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: <path to venv>/venv/bin/python3 /tmp/pip-standalone-pip-5xbqguwg/ install --ignore-installed --no-user --prefix /tmp/pip-build-env-3b3hrzfg/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i -- 'flit_core >=2,<3'
       cwd: None
  Complete output (67 lines):
  Collecting flit_core<3,>=2
    Using cached flit_core-2.3.0.tar.gz (22 kB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: <path to venv>/venv/bin/python3 /tmp/pip-standalone-pip-5xbqguwg/ get_requires_for_build_wheel /tmp/tmpogp2e7mb
         cwd: /tmp/pip-install-71xy4ixf/flit-core_156db09074064d12968592391ad52e71
    Complete output (1 lines):
    <path to venv>/venv/bin/python3: can't find '__main__' module in '/tmp/pip-standalone-pip-5xbqguwg/'

^repeated error while it tries earlier versions of flit_core

Code of Conduct

@g8trbryant g8trbryant added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Apr 26, 2021
Copy link

uranusjr commented Apr 27, 2021

This is a side effect of pip 21.1 switching to use zipapp for PEP 517 dependency population (#9689). This crashes 3.6 in “PEP 517 inception” scenarios[see note] since pep517 falls back to “dumb” __file__ manipulation when importlib.resources is not available. I should have noticed this 😞

I think there are two solutions:

  1. Fall back to the “old” way of self-invocation on 3.6. This means people on 3.6 won’t receive Run pip in isolated env by building zip #9689, but it was already only fixing a fairly obscure issue anyway.
  2. Vendor importlib-resources (backport of the stdlib module) and patch pep517 to use it instead of the stdlib module.

Either way, the workaround will only be short-term, since we will be dropping 3.6 in a year.

Note: This will not crash without --no-binary, even if a package is sdist-only, as long as the build dependency has a wheel. When pep517 is invoked to build the sdist, is a plain file, so everything works. With --no-binary, build dependencies required by the sdist are also only fetched by source only, and pep517 by that point lives in a zip, causing the issue.

@uranusjr uranusjr added this to the 21.1.1 milestone Apr 27, 2021
inmantaci added a commit to inmanta/inmanta-core that referenced this issue Apr 28, 2021
# Description

Downgrade pip dependency due to bug: pypa/pip#9878

# Self Check:

- [ ] ~~Attached issue to pull request~~
- [x] Changelog entry
- [ ] ~~Type annotations are present~~
- [ ] ~~Code is clear and sufficiently documented~~
- [ ] ~~No (preventable) type errors (check using make mypy or make mypy-diff)~~
- [ ] ~~Sufficient test cases (reproduces the bug/tests the requested feature)~~
- [ ] ~~Correct, in line with design~~
- [ ] ~~End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )~~

# Reviewer Checklist:

- [ ] Sufficient test cases (reproduces the bug/tests the requested feature)
- [ ] Code is clear and sufficiently documented
- [ ] Correct, in line with design
inmantaci added a commit to inmanta/inmanta-core that referenced this issue May 18, 2021
Bumps [pip]( from 21.0.1 to 21.1.1.
<p><em>Sourced from <a href="">pip's changelog</a>.</em></p>
<h1>21.1.1 (2021-04-30)</h1>
<h2>Deprecations and Removals</h2>
<li>Temporarily set the new &quot;Value for ... does not match&quot; location warnings level
to <em>DEBUG</em>, to hide them from casual users. This prepares pip 21.1 for CPython
inclusion, while pip maintainers digest the first intake of location mismatch
issues for the <code>distutils</code>-<code>sysconfig</code> transition. (<code>[#9912](pypa/pip#9912) &lt;;</code>_)</li>
<h2>Bug Fixes</h2>
<li>This change fixes a bug on Python <!-- raw HTML omitted -->`_)</li>
<li>Fix compatibility between distutils and sysconfig when the project name is unknown outside of a virtual environment. (<code>[#9838](pypa/pip#9838) &lt;;</code>_)</li>
<li>Fix Python 3.6 compatibility when a PEP 517 build requirement itself needs to be
built in an isolated environment. (<code>[#9878](pypa/pip#9878) &lt;;</code>_)</li>
<h1>21.1 (2021-04-24)</h1>
<li>Start installation scheme migration from <code>distutils</code> to <code>sysconfig</code>. A
warning is implemented to detect differences between the two implementations to
encourage user reports, so we can avoid breakages before they happen.</li>
<li>Add the ability for the new resolver to process URL constraints. (<code>[#8253](pypa/pip#8253) &lt;;</code>_)</li>
<li>Add a feature <code>--use-feature=in-tree-build</code> to build local projects in-place
when installing. This is expected to become the default behavior in pip 21.3;
see <code>Installing from local packages &lt;;</code>_
for more information. (<code>[#9091](pypa/pip#9091) &lt;;</code>_)</li>
<li>Bring back the &quot;(from versions: ...)&quot; message, that was shown on resolution failures. (<code>[#9139](pypa/pip#9139) &lt;;</code>_)</li>
<li>Add support for editable installs for project with only setup.cfg files. (<code>[#9547](pypa/pip#9547) &lt;;</code>_)</li>
<li>Improve performance when picking the best file from indexes during <code>pip install</code>. (<code>[#9748](pypa/pip#9748) &lt;;</code>_)</li>
<li>Warn instead of erroring out when doing a PEP 517 build in presence of
<code>--build-option</code>. Warn when doing a PEP 517 build in presence of
<code>--global-option</code>. (<code>[#9774](pypa/pip#9774) &lt;;</code>_)</li>
<h2>Bug Fixes</h2>
<li>Fixed <code>--target</code> to work with <code>--editable</code> installs. (<code>[#4390](pypa/pip#4390) &lt;;</code>_)</li>
<li>Add a warning, discouraging the usage of pip as root, outside a virtual environment. (<code>[#6409](pypa/pip#6409) &lt;;</code>_)</li>
<!-- raw HTML omitted -->
<p>... (truncated)</p>
<li><a href=""><code>c53d88c</code></a> Bump for release</li>
<li><a href=""><code>4417e7f</code></a> Update AUTHORS.txt</li>
<li><a href=""><code>0c29bfe</code></a> Merge pull request <a href="">#9912</a> from uranusjr/sysconfig-remove-warning-for-python-re...</li>
<li><a href=""><code>f56ec32</code></a> Make location mismatch messages DEBUG level</li>
<li><a href=""><code>999b121</code></a> Merge pull request <a href="">#9883</a> from uranusjr/isolated-pip-py36-compat</li>
<li><a href=""><code>f884203</code></a> Fallback to self-invoke via directory on 3.6</li>
<li><a href=""><code>7a77484</code></a> Merge pull request <a href="">#9835</a> from jamescurtin/9831-bugfix</li>
<li><a href=""><code>914bcc3</code></a> Merge pull request <a href="">#9838</a> from uranusjr/sysconfig-header-with-none-project</li>
<li><a href=""><code>2a009a0</code></a> Better explanatory comment</li>
<li><a href=""><code>e7b1722</code></a> Set dist_name to UNKNOWN when empty outside venv</li>
<li>Additional commits viewable in <a href="">compare view</a></li>
<br />

[![Dependabot compatibility score](](

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)


<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 28, 2021
@pradyunsg pradyunsg removed the S: needs triage Issues/PRs that need to be triaged label Mar 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
type: bug A confirmed bug or unintended behavior
None yet

Successfully merging a pull request may close this issue.

3 participants