pip install throws an error when empty Content-Type is returned by official pypi index

[Pio-kaw]

Description

For the pip install command I'm running, I have 2 PYPI repo indexes set up - as a main index, private mirror where all dependencies are downloaded from, and as an extra index - pypi.org in case of missing dependency / private mirror being down.

My build process consists of several builders running behind a company's VPN. Each builder might run concurrently several builds. Each build runs concurrently 10+ pip install (1 pip install for every venv that is created concurrently). Most dependencies that have their version locked are used from cache on the builder, so they are not redownloaded every time.

I have made sure that all dependencies used by me are available on my private index.

This issue seems to occur only for deps used for testing, where I do not have the lockfile generated for them. My assumption is that pip install tries to resolve the newest version for all subdependencies of my testing dependencies, because they are not locked. To find the newest version, it checks for available version in both indexes, because one of them might actually contain a newer version. I believe pypi.org (that is my extra index) is rate-limiting me in this case and empty Content-Type is received in the response sporadically, which causes it to crash, even though (main) private index gave it a valid version list for the said dependency. So basically:

1. Dependency "foo" is locked to a specified version. "foo" has "bar" in its requirements, which requires at least version X
2. Main index is checked, X version is available, no newer version here for "bar"
3. Extra index is checked (who knows, maybe X.0.1 version would be there which is newer and thus it should be taken), invalid response is received
4. pip does not know how to parse received response, warning is displayed that Content type "" is not valid
5. Pip throws an error that there is no dependency version that satisfies the requirement (from versions: none)

These errors were much more prominent when private index was not available in my build process yet, which further solidifies my speculation. They seem to be much more rare now that only some dev dependencies are checked in official pypi.org index. Rerunning the build pretty much always fixes the previous error. Even though it says "skipping page" I believe for some reason it does not skip it, or known versions from other indexes are not accounted for in this case.

I believe in this case, if main index returned proper list of dependency versions and the response received from the extra index is not valid, a warning should have been displayed that extra-index is not accounted for in this case. I suppose that "Warning: skipping page (...)" is supposed to do that, but it seems it does not actually fully skip it, it feels like pip in this case lost the information about available versions for this dependency received from the other mirror.

Expected behavior

Pip shows a warning that it could not resolve the dependency version from specified extra index, proceeds to use only working main index.

12:49:57  WARNING: Skipping page https://pypi.org/simple/black/ because the GET request got Content-Type: .The only supported Content-Type is text/html
12:49:57  ERROR: Could not find a version that satisfies the requirement black==21.7b0 (from versions: none)
12:49:57  ERROR: No matching distribution found for black==21.7b0

Other cases (all of them are followed by could not find a version from versions none error):

Skipping page https://pypi.org/simple/mypy/ because the GET request got Content-Type: .The only supported Content-Type is text/html

13:57:16  WARNING: Skipping page https://pypi.org/simple/lazy-loader/ because the GET request got Content-Type: .The only supported Content-Type is text/html

13:55:08  WARNING: Skipping page https://pypi.org/simple/attrs/ because the GET request got Content-Type: .The only supported Content-Type is text/html

pip version

22.0.4

Python version

3.8.8

OS

ubuntu 20.04

How to Reproduce

1. Set up private mirror and pypi.org as an extra mirror
2. Create requirements file that does not have reqs locked to specific version (i.e. don't use a lockfile)
3. Trigger big amount of pip install operations like specified in the description (some subdependencies might have specified minimal version, but not the maximum one).
4. This issue is sporadic and might need many reruns under a heavy load for a reproduction.

Output

(...)
12:49:57  WARNING: Skipping page https://pypi.org/simple/black/ because the GET request got Content-Type: .The only supported Content-Type is text/html
12:49:57  ERROR: Could not find a version that satisfies the requirement black==21.7b0 (from versions: none)
12:49:57  ERROR: No matching distribution found for black==21.7b0
12:49:57  WARNING: You are using pip version 22.0.4; however, version 23.0.1 is available.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

Duplicate of #11340.

[Pio-kaw]

Duplicate of #11340.

Thanks for the response @uranusjr.
The thing is, should it still affect me even though I'm not using pypi.org for packages as my main index? The issue seems to be pretty much gone since I've locked all my dependencies and switched to privately hosted index. It still occurs for requirement files where lockfile (with all subdependencies listed and locked to specific versions) is not generated.

I'm running single version of pip/python on my builders AFAIK. It seems to happen only when extra-index is used to check for any other versions, as I've said. For example I might specify that I want pytest==7.3.1, but that does not mean all dependencies of pytest are locked in as well. It might depend on pytest-metadata>=2.0.1 and because it is specified as >= (or even without any version at all, just pytest), it will mean that pip will crawl all indexes for the newest version of pytest-metadata. In this case, malformed response from pypi.org causes it to crash, even though it is extra-index, not the main index

[uranusjr]

The issue mainly comes from PyPI sometimes gives an erroneous HTTP response (actually the CDN PyPI uses). Your own server should not do that, and if it does you can fix the server.