Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing from source with --pre fails under specific condition with pip 21.2.1 #10222

Closed
t20100 opened this issue Jul 27, 2021 · 3 comments · Fixed by #10692
Closed

Installing from source with --pre fails under specific condition with pip 21.2.1 #10222

t20100 opened this issue Jul 27, 2021 · 3 comments · Fixed by #10692
Labels
C: PEP 517 impact Affected by PEP 517 processing type: bug A confirmed bug or unintended behavior

Comments

@t20100
Copy link
Contributor

t20100 commented Jul 27, 2021

Description

There is an issue when a project has a build dependency for which a pre-release version is available and which is stated both in pyproject.toml and setup_requires in setup.py.

In this case, trying to install it from source with pip (either from the source directory or from a tarball) with the --pre option fails (see below for the output).

This issue arises with cython for which an alpha version is available.

This issue does not occur:

  • When removing the failing dependency from either pyproject.toml or setup_requires (which is what I'll do).
  • Without the --pre flag
  • When also adding --no-build-isolation

Expected behavior

I'm not sure what is the expected behavior, but this was working with pip v21.1.3 and previous and breaks with pip v21.2.x

pip version

21.2.1 and 21.2

Python version

3.7

OS

macos

How to Reproduce

Create a project in a testpip directory with:

  • pyproject.toml:
    [build-system]
    requires = ["setuptools", "wheel", "Cython"]
    build-backend = "setuptools.build_meta"
    
  • setup.py:
    setup(
        name="testpip",
        version="0.0.1",
        py_module=["testpip"],
        setup_requires=["Cython"],
    )
    
  • testpip.py

Run pip install . --pre from testpip/

Output

pip install . --pre
Processing /testpip

  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Some build dependencies for file:///testpip conflict with the backend dependencies: Cython==3.0.0a9 is incompatible with Cython.


### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).
@t20100
Copy link
Contributor Author

t20100 commented Nov 26, 2021

https://github.com/pypa/pip/blob/main/src/pip/_internal/build_env.py#L182
This patch fixes this particular issue:

<                 if dist.version not in req.specifier:
---
>                 if not req.specifier.contains(dist.version, prereleases=True):

But I have no clue about the side effects.
Anyway, if this is on the right track, I can turn this into a PR.

@uranusjr
Copy link
Member

I believe your change is actually correct. The code around there only ensures the build environment is set up correctly, and should allow prereleases. Would you mind creating a pull request containing this change, and add a test to ensure this won’t break in the future?

@uranusjr uranusjr added C: PEP 517 impact Affected by PEP 517 processing and removed S: needs triage Issues/PRs that need to be triaged labels Nov 26, 2021
@t20100
Copy link
Contributor Author

t20100 commented Nov 26, 2021

Yes, sure!

t20100 added a commit to t20100/pip that referenced this issue Nov 29, 2021
t20100 added a commit to t20100/pip that referenced this issue Dec 6, 2021
blink1073 added a commit to jupyter-server/jupyter_server that referenced this issue May 6, 2022
inmantaci pushed a commit to inmanta/inmanta-core that referenced this issue Jul 21, 2022
Bumps [pip](https://github.com/pypa/pip) from 22.1.2 to 22.2.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>22.2 (2022-07-21)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove the <code>html5lib</code> deprecated feature flag. (<code>[#10825](pypa/pip#10825) &lt;https://github.com/pypa/pip/issues/10825&gt;</code>_)</li>
<li>Remove <code>--use-deprecated=backtrack-on-build-failures</code>. (<code>[#11241](pypa/pip#11241) &lt;https://github.com/pypa/pip/issues/11241&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>
<p>Add support to use <code>truststore &lt;https://pypi.org/project/truststore/&gt;</code>_ as an
alternative SSL certificate verification backend. The backend can be enabled on Python
3.10 and later by installing <code>truststore</code> into the environment, and adding the
<code>--use-feature=truststore</code> flag to various pip commands.</p>
<p><code>truststore</code> differs from the current default verification backend (provided by
<code>certifi</code>) in it uses the operating system’s trust store, which can be better
controlled and augmented to better support non-standard certificates. Depending on
feedback, pip may switch to this as the default certificate verification backend in
the future. (<code>[#11082](pypa/pip#11082) &lt;https://github.com/pypa/pip/issues/11082&gt;</code>_)</p>
</li>
<li>
<p>Add <code>--dry-run</code> option to <code>pip install</code>, to let it print what it would install but
not actually change anything in the target environment. (<code>[#11096](pypa/pip#11096) &lt;https://github.com/pypa/pip/issues/11096&gt;</code>_)</p>
</li>
<li>
<p>Record in wheel cache entries the URL of the original artifact that was downloaded
to build the cached wheels. The record is named <code>origin.json</code> and uses the PEP 610
Direct URL format. (<code>[#11137](pypa/pip#11137) &lt;https://github.com/pypa/pip/issues/11137&gt;</code>_)</p>
</li>
<li>
<p>Support <code>PEP 691 &lt;https://peps.python.org/pep-0691/&gt;</code><em>. (<code>[#11158](pypa/pip#11158) &lt;https://github.com/pypa/pip/issues/11158&gt;</code></em>)</p>
</li>
<li>
<p>pip's deprecation warnings now subclass the built-in <code>DeprecationWarning</code>, and
can be suppressed by running the Python interpreter with
<code>-W ignore::DeprecationWarning</code>. (<code>[#11225](pypa/pip#11225) &lt;https://github.com/pypa/pip/issues/11225&gt;</code>_)</p>
</li>
<li>
<p>Add <code>pip inspect</code> command to obtain the list of installed distributions and other
information about the Python environment, in JSON format. (<code>[#11245](pypa/pip#11245) &lt;https://github.com/pypa/pip/issues/11245&gt;</code>_)</p>
</li>
<li>
<p>Significantly speed up isolated environment creation, by using the same
sources for pip instead of creating a standalone installation for each
environment. (<code>[#11257](pypa/pip#11257) &lt;https://github.com/pypa/pip/issues/11257&gt;</code>_)</p>
</li>
<li>
<p>Add an experimental <code>--report</code> option to the install command to generate a JSON report
of what was installed. In combination with <code>--dry-run</code> and <code>--ignore-installed</code> it
can be used to resolve the requirements. (<code>[#53](pypa/pip#53) &lt;https://github.com/pypa/pip/issues/53&gt;</code>_)</p>
</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix <code>pip install --pre</code> for packages with pre-release build dependencies defined
both in <code>pyproject.toml</code>'s <code>build-system.requires</code> and <code>setup.py</code>'s
<code>setup_requires</code>. (<code>[#10222](pypa/pip#10222) &lt;https://github.com/pypa/pip/issues/10222&gt;</code>_)</li>
<li>When pip rewrites the shebang line in a script during wheel installation,
update the hash and size in the corresponding <code>RECORD</code> file entry. (<code>[#10744](pypa/pip#10744) &lt;https://github.com/pypa/pip/issues/10744&gt;</code>_)</li>
<li>Do not consider a <code>.dist-info</code> directory found inside a wheel-like zip file
as metadata for an installed distribution. A package in a wheel is (by</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/8e7e76e60f4e115ea1201bee2f176377a718fce1"><code>8e7e76e</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/b6f6a94e36f10a4535ea5bbdc6b351f62003eede"><code>b6f6a94</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/790725aca3f60c745e33827a6079d9600da373d8"><code>790725a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11274">#11274</a> from sbidoul/install-report-note-sbi</li>
<li><a href="https://github.com/pypa/pip/commit/d4b9e187aa7cc5ab14b2339f6171f7f2ea6504e9"><code>d4b9e18</code></a> Add clarifications to the installation report documentation</li>
<li><a href="https://github.com/pypa/pip/commit/b1a01ef762a78af1194958a1c874015eaf81fd04"><code>b1a01ef</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11265">#11265</a> from finnagin/main</li>
<li><a href="https://github.com/pypa/pip/commit/48bcb0a4ccd30a9d00e58fe58827772e307a7e39"><code>48bcb0a</code></a> reformat to pass pre-commit check</li>
<li><a href="https://github.com/pypa/pip/commit/a7c1fe3bff5655393018c53b448b669b3525515b"><code>a7c1fe3</code></a> Remove utc fixture from tests</li>
<li><a href="https://github.com/pypa/pip/commit/0c574f72905185d62bcca741c813df9bae1d9282"><code>0c574f7</code></a> Remove time import</li>
<li><a href="https://github.com/pypa/pip/commit/246fef19149eea893f1cf3efd53f9b17c94c952f"><code>246fef1</code></a> Remove utc fixture</li>
<li><a href="https://github.com/pypa/pip/commit/c9cb7f4629bdd8c61b792feff6dacb1d2e848d57"><code>c9cb7f4</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11270">#11270</a> from uranusjr/upgrade-pre-commit-hooks</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/22.1.2...22.2">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=22.1.2&new-version=22.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>
inmantaci pushed a commit to inmanta/inmanta-core that referenced this issue Jul 21, 2022
Bumps [pip](https://github.com/pypa/pip) from 22.1.2 to 22.2.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>22.2 (2022-07-21)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove the <code>html5lib</code> deprecated feature flag. (<code>[#10825](pypa/pip#10825) &lt;https://github.com/pypa/pip/issues/10825&gt;</code>_)</li>
<li>Remove <code>--use-deprecated=backtrack-on-build-failures</code>. (<code>[#11241](pypa/pip#11241) &lt;https://github.com/pypa/pip/issues/11241&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>
<p>Add support to use <code>truststore &lt;https://pypi.org/project/truststore/&gt;</code>_ as an
alternative SSL certificate verification backend. The backend can be enabled on Python
3.10 and later by installing <code>truststore</code> into the environment, and adding the
<code>--use-feature=truststore</code> flag to various pip commands.</p>
<p><code>truststore</code> differs from the current default verification backend (provided by
<code>certifi</code>) in it uses the operating system’s trust store, which can be better
controlled and augmented to better support non-standard certificates. Depending on
feedback, pip may switch to this as the default certificate verification backend in
the future. (<code>[#11082](pypa/pip#11082) &lt;https://github.com/pypa/pip/issues/11082&gt;</code>_)</p>
</li>
<li>
<p>Add <code>--dry-run</code> option to <code>pip install</code>, to let it print what it would install but
not actually change anything in the target environment. (<code>[#11096](pypa/pip#11096) &lt;https://github.com/pypa/pip/issues/11096&gt;</code>_)</p>
</li>
<li>
<p>Record in wheel cache entries the URL of the original artifact that was downloaded
to build the cached wheels. The record is named <code>origin.json</code> and uses the PEP 610
Direct URL format. (<code>[#11137](pypa/pip#11137) &lt;https://github.com/pypa/pip/issues/11137&gt;</code>_)</p>
</li>
<li>
<p>Support <code>PEP 691 &lt;https://peps.python.org/pep-0691/&gt;</code><em>. (<code>[#11158](pypa/pip#11158) &lt;https://github.com/pypa/pip/issues/11158&gt;</code></em>)</p>
</li>
<li>
<p>pip's deprecation warnings now subclass the built-in <code>DeprecationWarning</code>, and
can be suppressed by running the Python interpreter with
<code>-W ignore::DeprecationWarning</code>. (<code>[#11225](pypa/pip#11225) &lt;https://github.com/pypa/pip/issues/11225&gt;</code>_)</p>
</li>
<li>
<p>Add <code>pip inspect</code> command to obtain the list of installed distributions and other
information about the Python environment, in JSON format. (<code>[#11245](pypa/pip#11245) &lt;https://github.com/pypa/pip/issues/11245&gt;</code>_)</p>
</li>
<li>
<p>Significantly speed up isolated environment creation, by using the same
sources for pip instead of creating a standalone installation for each
environment. (<code>[#11257](pypa/pip#11257) &lt;https://github.com/pypa/pip/issues/11257&gt;</code>_)</p>
</li>
<li>
<p>Add an experimental <code>--report</code> option to the install command to generate a JSON report
of what was installed. In combination with <code>--dry-run</code> and <code>--ignore-installed</code> it
can be used to resolve the requirements. (<code>[#53](pypa/pip#53) &lt;https://github.com/pypa/pip/issues/53&gt;</code>_)</p>
</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix <code>pip install --pre</code> for packages with pre-release build dependencies defined
both in <code>pyproject.toml</code>'s <code>build-system.requires</code> and <code>setup.py</code>'s
<code>setup_requires</code>. (<code>[#10222](pypa/pip#10222) &lt;https://github.com/pypa/pip/issues/10222&gt;</code>_)</li>
<li>When pip rewrites the shebang line in a script during wheel installation,
update the hash and size in the corresponding <code>RECORD</code> file entry. (<code>[#10744](pypa/pip#10744) &lt;https://github.com/pypa/pip/issues/10744&gt;</code>_)</li>
<li>Do not consider a <code>.dist-info</code> directory found inside a wheel-like zip file
as metadata for an installed distribution. A package in a wheel is (by</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/8e7e76e60f4e115ea1201bee2f176377a718fce1"><code>8e7e76e</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/b6f6a94e36f10a4535ea5bbdc6b351f62003eede"><code>b6f6a94</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/790725aca3f60c745e33827a6079d9600da373d8"><code>790725a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11274">#11274</a> from sbidoul/install-report-note-sbi</li>
<li><a href="https://github.com/pypa/pip/commit/d4b9e187aa7cc5ab14b2339f6171f7f2ea6504e9"><code>d4b9e18</code></a> Add clarifications to the installation report documentation</li>
<li><a href="https://github.com/pypa/pip/commit/b1a01ef762a78af1194958a1c874015eaf81fd04"><code>b1a01ef</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11265">#11265</a> from finnagin/main</li>
<li><a href="https://github.com/pypa/pip/commit/48bcb0a4ccd30a9d00e58fe58827772e307a7e39"><code>48bcb0a</code></a> reformat to pass pre-commit check</li>
<li><a href="https://github.com/pypa/pip/commit/a7c1fe3bff5655393018c53b448b669b3525515b"><code>a7c1fe3</code></a> Remove utc fixture from tests</li>
<li><a href="https://github.com/pypa/pip/commit/0c574f72905185d62bcca741c813df9bae1d9282"><code>0c574f7</code></a> Remove time import</li>
<li><a href="https://github.com/pypa/pip/commit/246fef19149eea893f1cf3efd53f9b17c94c952f"><code>246fef1</code></a> Remove utc fixture</li>
<li><a href="https://github.com/pypa/pip/commit/c9cb7f4629bdd8c61b792feff6dacb1d2e848d57"><code>c9cb7f4</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11270">#11270</a> from uranusjr/upgrade-pre-commit-hooks</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/22.1.2...22.2">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=22.1.2&new-version=22.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C: PEP 517 impact Affected by PEP 517 processing type: bug A confirmed bug or unintended behavior
Projects
None yet
2 participants