Subcommand to display all "top level" requirements

[notatallshaw]

What's the problem this feature will solve?

This subcommand would be useful for wrapping Pip in a script or debugging user issues.

Describe the solution you'd like

Given a command like pip requirements ... where the ... is the usual ways to provide requirements like you would with download or install subcommands, Pip would collect all the local requirements it normally does and print them out.

Alternative Solutions

• Do not provide
• Could be a flag of existing subcommands instead

Additional context

Came up while discussing #12160.

With this feature implemented it would be significantly more trivial to make a third party package that implemented the behavior described by wrapping Pip and calling it from a subprocess instead of hacking the internal Pip API.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

Closing due to no expressed interest from Pip maintainers.

IMO without this it's not really possible to write useful tools that call Pip from the command line to do custom resolution tasks.

So instead one has to import Pip directly in Python, which is fragile but the only practical solution, as we can see from the pip-tools repo,

[sbidoul]

@notatallshaw the Json installation report does mention top level requirements. Combined with --dry-run and --no-deps does this help with your use case?

[notatallshaw]

@sbidoul Not really, that still downloads wheels and also builds sdists

[sbidoul]

that still downloads wheels and also builds sdists

True. That is solvable, though and there is ongoing work in that direction.

On second read, I think I don't quite understand why this use case is important. If a tool can call pip requirements ... it knows what requirements are in ... and these are the top level ones?

[pfmoore]

I agree with @sbidoul - how is this anything more than a requirements file parser?

[pfmoore]

Update: in the original issue it was me who suggested a subcommand. But that was prefaced with the condition “if it’s so complicated…”. But I still don’t believe it is complicated - it’s just a list of command line arguments and requirement files, which can be parsed very simplistically. There’s no need for the full complexity of pip’s parsing or option handling.

[notatallshaw]

Pip can source top level requirements from the command line and requirement files simultaneously. There can be multiple requirements files, they can be nested infinitely and use other weird options, the behavior may change in the future as new features are added.

Only Pip can reproduce it's own behavior, that's why wrapping tools import Pip directly.

[pfmoore]

🤷 The use case for this was "write a script to install a set of requirements one by one". That script doesn't need to support all of the complexity that pip supports.

Is there another use case that I'm missing?

To put this another way, you are correct that there's no interest from the pip maintainers (at least, not from me) for exposing pip's command line syntax, and the parsing of that syntax, as a service for other tools to use. I would absolutely not hold pip's CLI syntax up as a good example for other tools to copy - quite the opposite, I'd strongly encourage them to design their own UI using a more modern framework than optparse.

And nesting and "weird options" in requirement files are just another aspect of requirements files not being standardised. We should standardise something rather than encouraging more extensive use of requirements files.

[notatallshaw]

As you say I already closed this, I don't understand why there are follow up questions after it's closed. As you probably know I'm extremely familiar with Pip and understand the viability of various solutions for my use cases.

For me the effort to justify this is greater than importing Pip's internal functions and pinning my Pip version, like pip-tools does.

The only consequence is it means I won't share any of my tools publicly because I can't depend on any interface Pip has to the extent I would want to support other users who would benefit from my tooling.

[pfmoore]

I don't understand why there are follow up questions after it's closed.

Mostly because you closed it saying there was no interest from the pip maintainers, which had 2 effects (for me, at least):

1. It reminded me this issue even existed. I only see things if I get an email because of activity, so if there's an initial issue and no followup from anyone, it tends to fall off my radar.
2. I didn't want you to feel your suggestion had been ignored, so I wanted to both clarify why I didn't think this was worth it, and check you didn't have other use cases that you hadn't mentioned.

Plus, of course, once a discussion starts, I get emails, so I tend to follow up more 🙂

Anyway, I don't want to waste anyone's time with extended discussion on a closed issue, so I'll stop here.