Command to know what file would be downloaded for a requirement

[uranusjr]

What's the problem this feature will solve?

I want to know what version (and potentially ahich artifact) would be downloaded/installed when I pip install a package. Currently I do something like pip download --no-deps --dest %TMP% and read the output, but that can take a long time if the package is large (e.g. numpy).

Describe the solution you'd like

Something like pip find numpy to only execute the finding part, but instead of downloading (and inspecting) the package, print the link that would be downloaded instead:

$ py -3.7 -m pip find numpy
https://files.pythonhosted.org/packages/4e/9d/c129d78e6b942303b762ccfdf1f8339de80c5e6021b14ef0c99ec5bdc6aa/numpy-1.16.3-cp37-cp37m-win_amd64.whl

Alternative Solutions

My current approach is described above. Alternative design includes pip install --dry-run, but that does not feel right because it implies dependency resolution (which requires the package to be downloaded).

An alternative output is to list all applicable links, instead of only the best match.

Additional context

A command that automatically download packages (to a temporary location) and produce a requirements.txt could be useful as well, but I can write a script using existing pip commands easily enough, so pip don’t really need to provide that functionality.

[dstufft]

I don't think it's entirely possible (and I think the initial snippet is wrong) once we get a real version solver in place. It's entirely possible that pip install foo will traverse the dependency graph and find that the latest version of foo is being excluded for some reason by one of it's dependencies (or we cannot install the latest version because of a conflict) and an older version will instead be installed.

However, the hypothetical pip find foo that does not do dependency resolution will just grab the latest version that matches the specifier, which would be different than what pip install foo would do in the above scenario.

[uranusjr]

Good point, I did not think of that. How about defining this as “outputting all possible links for this specification” instead? This don’t need to guarentee what would be actually installed (or anything at all), only list the links found.

My use case is exactly to feed the output to a resolver, but I don’t want to re-implement all the configuration and edge cases in pip.

[di]

In #6740 I made a similar request:

What's the problem this feature will solve?
Currently there is no way for third-party projects to use PackageFinder's logic via the command line. Instead, projects like pip-compile and pipenv need to import from the _internal API.

Describe the solution you'd like
Ideally this would mimic PackageFinder.find_all_candidates() as much as possible, and would have a human-friendly and machine-friendly format. Something like:

$ pip find bar
bar    0.1.0    https://files.pythonhosted.org/packages/2c/85/d6397edeceb1c81b5b99b1046033c8854abac3e50971c8d2915e63ce54c0/bar-0.1.0.tar.gz#sha256=6a711b5b06a662fe3f8e897310456b02127fb15d89310161fc8af7f415e270a1 (from https://pypi.org/simple/bar/)
bar    0.2.0    https://files.pythonhosted.org/packages/ba/bf/81ef8a2c80f4f86e9e6570605b7b5a277950cbd0ff9be901633159f4110e/bar-0.2.0.tar.gz#sha256=4a97e8664a22eec73f65d522cc36b4c693b61f0557760a856bb18a8fb67a8881 (from https://pypi.org/simple/bar/)
bar    0.2.1    https://files.pythonhosted.org/packages/77/d6/760c7a0ee7385ba4b0abb1d38af97e8c2f34cb47fe301315e48204088047/bar-0.2.1.tar.gz#sha256=b0a71c865729f9e8ac0f74e666b46c5ed1f5addac3425be7e06ca4828cec90ef (from https://pypi.org/simple/bar/)

$ pip find bar --format json
[
  {
    "name": "bar",
    "version": "0.1.0",
    "link": "https://files.pythonhosted.org/packages/2c/85/d6397edeceb1c81b5b99b1046033c8854abac3e50971c8d2915e63ce54c0/bar-0.1.0.tar.gz#sha256=6a711b5b06a662fe3f8e897310456b02127fb15d89310161fc8af7f415e270a1",
    "index": "https://pypi.org/simple/bar/"
  }, {
    "name": "bar",
    "version": "0.2.0",
    "link": "https://files.pythonhosted.org/packages/ba/bf/81ef8a2c80f4f86e9e6570605b7b5a277950cbd0ff9be901633159f4110e/bar-0.2.0.tar.gz#sha256=4a97e8664a22eec73f65d522cc36b4c693b61f0557760a856bb18a8fb67a8881",
    "index": "https://pypi.org/simple/bar/"
  }, {
    "name": "bar",
    "version": "0.2.1",
    "link": "https://files.pythonhosted.org/packages/77/d6/760c7a0ee7385ba4b0abb1d38af97e8c2f34cb47fe301315e48204088047/bar-0.2.1.tar.gz#sha256=b0a71c865729f9e8ac0f74e666b46c5ed1f5addac3425be7e06ca4828cec90ef",
    "index": "https://pypi.org/simple/bar/"
  }
]

Alternative Solutions

• Splitting out the PackageFinder logic into a separate package (Splitting out PackageFinder logic #5800) and having third-parties use this instead
• Refactor pip search to use PackageFinder ("pip search" should use PackageFinder, not XML-RPC API #395) and add extra options there to overload the current behavior/API of pip search

Additional context
Recent internal API changes in 19.2 will break pip-compile: jazzband/pip-tools#853

[cjerdonek]

FYI, I posted PR #6910, which makes progress towards this.

[cjerdonek]

Also, FWIW, re: the original issue, I like the idea of a pip find command.

[cjerdonek]

FYI PR #7016, which was just posted, relates to this.

[ziebam]

Similarly to #7975, I think this issue was solved by #8978 and further improvements are currently tracked #10052.

Please excuse my nit-pickiness, just checking out the issues that CodeTriage sends me. 😅

[uranusjr]

Neither pip index version now and #10052 covers what I originally needed. I was looking for the information “what file will be downloaded when I install this requirement”. Both features you linked only operate at the version level, but I need the information at the file level. Numpy 1.16.3, for example, contains 22 wheels and 1 sdist, I want to know which one will be selected.

[pradyunsg]

Is this a subset of the broader "do a dry-run, and tell me what you would've done" ALA #53?

[uranusjr]

Yes if we provide good enough dry-run logging (I want to parse the filename in a script).

[sbidoul]

Do we reckon that this is adequately resolved by #10771 (pip install --dry-run --report), together with #11111 (PEP 658) ?

[pradyunsg]

I think so, yes.