pip install is downloading pre-release version without the --pre argument

[ghost]

• Pip version:10.0.1
• Python version:3.6.5
• Operating system:GNU/Linux (openSUSE Tumbleweed)

Dear all

I have created a new gnuhealth pre-release package using PEP440 versioning (3.3a1).

The problem is that if someone types "pip install --user gnuhealth" (without explicitly passing --pre as an argument) it will download the pre-release unstable version that is intended for developers and not for general users.

Any thoughts are most welcome.

Best
Luis

[benoit-pierre]

I think this is normal as it's the only version available on PyPi; see PEP 440.

[ghost]

Thanks !

I might be wrong, but IMHO, I don't think this should not be the behavior. A pre-release, regardless whether of the number of existing versions, should only be retrievable when using the --pre argument on pip. Otherwise, regular users will download it and get an unstable version.

Best

[benoit-pierre]

But that's not what PEP 440 says:

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

(Emphasis mine).

[ghost]

hmm. I am not passing any version specifier as an argument to pip, but probably is not referring to this. In testpypi there are two pre-release versions and still behaves the same way.

[benoit-pierre]

I'm not seeing an issue when using test.pypi:

> pip download --index-url https://test.pypi.org/simple gnuhealth
Looking in indexes: https://test.pypi.org/simple
Collecting gnuhealth
  Downloading https://test-files.pythonhosted.org/packages/ae/aa/d04040b251f9edef6ee3b949ddf964a7568d17362738d6830515e5a43eac/gnuhealth-3.3.2.tar.gz (780kB)
    100% |████████████████████████████████| 788kB 10.5MB/s 
  Saved ./gnuhealth-3.3.2.tar.gz
Collecting pytz (from gnuhealth)
  Could not find a version that satisfies the requirement pytz (from gnuhealth) (from versions: )
No matching distribution found for pytz (from gnuhealth)

[ghost]

Thank you for the feedback !

Yes, you are right. I double checked on testpypi, and this is how it seems to work:

• If there is a non pre-release version of the package on pypi, then pip works as expected, ignoring the pre-releases.
• if the only versions on pypi are pre-releases, it will choose the latest pre-release, even if no "--pre" argument is provided.

We still have the problem. Regular users will download an unstable version of a package, which is not ideal. I don't get the PEP440 rationale on this.

Best,

[pfmoore]

I'd have thought the rationale is simple - if I ask for X, I should get X if at all possible. Ideally, a stable release, but a pre-release is better than nothing at all. If you don't want general users to install your pre-release package before you release your first stable version, you should probably simply not publish it on PyPI.

[ghost]

Hi there !
I don't agree.
IMHO, development / pre-releases are not necessarily better than nothing at all. They are made for developers, not end users. They are unstable and they might contain (more) security issues. They are not ready to be used.

Forcing the developer to explicitly ask for a pre-release, using the "--pre" argument in all scenarios should be the best and easiest approach. That's something that the developer knows, and the end user won't get the wrong package.

[pradyunsg]

I don't know if this is something that had been brought up when PEP 440 was being discussed. @meanmicio perhaps you can try to check if that's indeed the case (the discussions are archived on distutils-sig and you can see when the PEP was accepted on the PEP page).

Moreover, if pip's implementation has to be changed, we have to update PEP 440 before that. That's going to need to go through distutils-sig as that's where packaging related PEPs are discussed.

[pfmoore]

I agree with @pradyunsg - if you want this to change, you'll have to start by proposing a change to PEP 440 (which would be discussed on distutils-sig, not here). If (and only if) that change is accepted and implemented, then pip would change to follow the amended standard.

[techalchemy]

(Sorry for butting in) -- just to comment from our perspective (we had discussed how to handle implementation separately -- have not yet implemented it) - it makes sense that if a user requests to have something installed, and there only exists a prerelease version, you give them that. See the discussion here (which basically amounts to -- the PEP, pip, and warehouse all work this way, so should we): pypa/pipenv#2022 (comment)

[ghost]

WARNING! : Definitely logical and consistent content

I got bit by this bug on Win10 with tensorflow. Yay!

>pip install scipy
… installing (release/expected version) of scipy

>pip install tensorflow
… here, have a rc version. why this happened? See reason1.

Go to reason1 : - Link

[ghost]

This would have been better. For anyone who expects sane/intuitive defaults for mundane tasks; that need to be automated.

From pypa/pipenv#2022 (comment)

perhaps with a printed warning at install time about only pre-releases being available for that particular dependency.

always with a printed warning before downloading the file, about only ...

[ghost]

If you totally agree with issue #5503 (comment) ; I have a new project that you'll like. Just 😕 & 🚀 this post if you want a link. I'll put it afterwards in the 'Project Link' (expandable).

Project Link

To be edited

[ghost]

https://www.python.org/dev/peps/pep-0440/#handling-of-pre-releases may not be wrong.

[ghost]

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

version specifier

1. What is a version specifier. And what's not. With examples. Not included in PEP 440.
   To me:- The command pip install tensorflow has no version specifier as the word tensorflow does not have a explicit and constant version number in it. And pip install is just the command.

[ghost]

By default, dependency resolution tools SHOULD:

• accept already installed pre-releases for all version specifiers
• accept remotely available pre-releases for version specifiers where there is no final or post release that satisfies the version specifier
• exclude all other pre-releases from consideration

SHOULD
2) So its not must.

exclude all other pre-releases
3) Pre-releases includes release candidates and all prior versions.

remotely available pre-releases for version specifiers
4) Well, there's no version specifier.

[ghost]

click to expand

I guess this hinges on version specifier. But I think PEP440 does not tell pip what to do.

PEP508 agrees with me & the users on what version specifier does not mean.

https://www.python.org/dev/peps/pep-0508/#names

Python distribution names are currently defined in PEP-345 [2]. Names act as the primary identifier for distributions. They are present in all dependency specifications, and are sufficient to be a specification on their own.

https://www.python.org/dev/peps/pep-0345/#version leads to PEP440.

https://www.python.org/dev/peps/pep-0345/#name

The name of the distributions.
Example:
Name: BeagleVote

For example it would be tensorflow with Name: TensorFlow in the metadata for the package.

---

I guess it's PEP 508+345+440+
pypa/pipenv#2022 (comment) & #5503 (comment) that help me understand this oversight.

5. pip install is downloading pre-release version without the --pre argument #5503 (comment) = the PEP says this, so it makes sense. I agree.

---

6. Installion Command with --pre and Pinning is Inconsistent pipenv#2022 (comment) is on a separate conglomerate of issues that is pipenv/issues/Multiple indexes #2022. Might not be relevant to this issue. But let's try to apply it here just for fun? Someone ping ncoghlan first.

7. Quoting pip install is downloading pre-release version without the --pre argument #5503 (comment)

it makes sense that if a user requests to have something installed, and there only exists a prerelease version, you give them that.

I know that pressure builds on developers. Some people may want the convenience, but does it fall into the general expectations? For eg, if you tried:-

• apt install wget
• pip install numpy
• search download Python and click on https://www.python.org/downloads/ what do you expect to get as the first link for Python?

release-candidate/beta/alpha/dev or latest-version. I expect latest-version. No pre-release version.
This way it's not pip that's at fault if tensorflow is having trouble with python compatibility.

I'll try to see if pipenv protects an environment from pip install tensorflow. If it does, it will be hilarious.
Edit1: Pipenv does the same on a new environment.
Edit2: This is on latest Win10 1809, python3.7.2, pip pip 19.0.2

[ghost]

Tensorflow is safe to use since stable release v1.13 . But the issue still remains unresolved.

[kohver]

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

Let's consider we have a package with the following versions (all are pre-releases):

0.0.1a1
0.0.2a1

And if I run pip install my-package I'll get the 0.0.2a1 version, since it's "the only available version that satisfies the version specifier".

But if I run pip install my-package!=0.0.2a1 I'll get the "No matching distribution found" error. However, the 0.0.1a1 is "the only available version that satisfies the version specifier", isn't it?

I see a controversy here.

[devinrsmith]

I'm running into this as well. IMO, it would be better to error out than to install pre-release software by default. I'd like to install the latest normal release, binary only, without needing to specify a version. Here's a quick rundown of what I see in a venv:

$ pip --version
pip 23.0.1 from /tmp/pip-5503/lib64/python3.11/site-packages/pip (python 3.11)

$ python --version
Python 3.11.2

$ pip install --only-binary=:all: connectorx
Collecting connectorx
  Using cached connectorx-0.3.2_alpha.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (47.1 MB)
Installing collected packages: connectorx
Successfully installed connectorx-0.3.2a2

In this specific case, it looks like since I'm on Python 3.11, and the latest normal release for connectorx (3.1) does not contain binary wheels for python 3.11, it decides to install the pre-release version which does contain Python 3.11 binary wheels.

Ideally, this command would error out by default. Since that's not the case today, is there a flag I can specify to explicitly turn off all form of pre-release wheels? --no-pre?

Note: on Python 3.10, connectorx 3.1 is correctly installed.

[devinrsmith]

It looks like PEP440 says tools SHOULD allow this alternative behavior:

Dependency resolution tools SHOULD also allow users to request the following alternative behaviours:
   ...
   excluding pre-releases for all version specifiers (reporting an error or warning if a pre-release is already installed locally, or if a pre-release is the only way to satisfy a particular specifier)

IMO, this could be satisfied by a --no-pre flag or some other form of configuration.