-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Long-term plan for parse_requirements
API
#121
Comments
@di fyi this is in https://github.com/nexB/pip-requirements I did start from the pip code and toned it down keeping the filtered line/blame-level history. I removed all the parts that are not about parsing and all network-related code, plus anything that would require accessing the local file system except for an optional nested files -r/-c loading and parsing. I found it a bit easier than starting from your fork (even though this may be very similar initial pip code). I kept and adapted all pip tests plus added tests from many requirements parsing tools (including yours ;) ) and eventually improved things such that:
The code is a bit quirky because it is essentially pip's own code adapted and mildly improved.
Anything like this would be great (any code that could parse pip requirements correctly, I am not talking about my experiment in particular). But there are really a set of pip-only things that IMHO do not fit in packaging
That would work, but the original parsing code is seriously tangled with mixed up CLI options handling, (random) file systems access and network calls. Having done the work above, I'd say that some parser could be surgically implanted back but I have a hard time to figure if and how pip's own parser could be seriously operated and exposed without breaking other pip features. FWIW, my use case is integration in ScanCode toollkit to detect, parse and collect any requirement files (it does also installed wheels, setup.py and other manifests) and replace Beyond this, we are implementing a generic version range (which will feed a basic dependency resolver) with https://github.com/nexB/univers to resolve dependent ranges in deps and in vulnerable version ranges. The whole goal of this is to support vulnerabilities data aggregation and vulnerability queries in https://github.com/nexb/vulnerablecode using the Package URLs found by scancode and possible resolved versions. |
Just to clarify that in my use cases the place where I parse requirements:
|
The
parse_requirements
API is currently a bit of a wart on this project as it doesn't adhere to the general goal of using the underlyingpip
CLI as much as possible, and instead is mostly a reimplementation of requirements parsing inpip
.As I see it we have three options:
pip
;pip
that exposes this functionalitypypa/packaging
as a utility library that could be reused bypip
.Short term, I think we should generally do 1 to unblock downstream work, with the understanding that this isn't the long-term goal.
For 2, @pombredanne mentioned working on a standalone requirements parser, but I'm not sure what the status of that is, perhaps we could get a summary here.
I think 3 would be interesting but it's longer-term work, and I'm not sure if the
pip
team would be receptive to it due to what I imagine is a limited use case.Ultimately I think 4 is the right answer here. This could involve upstreaming either the parsing logic here, in
pip
, in some other requirements parser, or a combination of all of them.The text was updated successfully, but these errors were encountered: