Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What does a URL constraint mean? #8253

Closed
pradyunsg opened this issue May 17, 2020 · 42 comments
Closed

What does a URL constraint mean? #8253

pradyunsg opened this issue May 17, 2020 · 42 comments
Labels
C: constraint Dealing with "constraints" (the -c option) C: direct url Direct URL references (PEP 440, PEP 508, PEP 610) state: needs discussion This needs some more discussion type: feature request Request for a new feature type: maintenance Related to Development and Maintenance Processes type: question User question

Comments

@pradyunsg
Copy link
Member

@pradyunsg pradyunsg commented May 17, 2020

Carrying forward discussion from #8210


To discuss about a constraints.txt file with the following content:

packaging @ git+https://github.com/pypa/packaging@20.1
  1. Should this be allowed?
  2. What functionality does this provide?
  3. Is this the best way to provide it?
@triage-new-issues triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label May 17, 2020
@pradyunsg pradyunsg added C: constraint Dealing with "constraints" (the -c option) state: needs discussion This needs some more discussion type: feature request Request for a new feature type: maintenance Related to Development and Maintenance Processes type: question User question labels May 17, 2020
@triage-new-issues triage-new-issues bot removed S: needs triage Issues/PRs that need to be triaged labels May 17, 2020
@uranusjr
Copy link
Member

@uranusjr uranusjr commented May 17, 2020

To me, this means whenever packaging is requested, use this link I provided. Further version specifiers should still apply (and fail resolution if they are not satisfied by the link target). The behaviour would be similar to

pip install <packages-i-want> "packaging @ git+https://github.com/pypa/packaging@20.1"

but without packaging being marked as user_requested (unless it is listed in <packages-i-want>, of course).

@dhellmann
Copy link

@dhellmann dhellmann commented May 18, 2020

I think of it as constraining the location from which distributions can be downloaded, just as a version expression like >=20.1 would constrain the version(s) that could be downloaded.

@pfmoore
Copy link
Member

@pfmoore pfmoore commented May 29, 2020

OK. I have done a more detailed investigation into the way that we'd implement NAME @ URL style constraints. The idea in principle is straightforward, but it has a number of interactions with other features of pip that make the overall behaviour far more complex.

The current implementation will have behaviours for all of these points that arise "by accident" as a result of the implementation choice to treat constraints as "just requirements that we don't install". The new resolver doesn't have that option, because the underlying model is completely different.

It's also true that there are likely reasonable choices to be made as to "how should this interaction work". It's not necessarily hard to choose a "correct" behaviour for any of the following points. But the implementation is decidedly non-trivial in the new resolver's existing model, so the cost of these features is disproportionately high, and we don't really have any idea at the moment if there's any actual use cases for these sorts of interactions (that's actual use cases as opposed to "yes I can see how this might be useful" 🙂)

  • Multiple URLs - should they be allowed? What would they mean? After all we can only get a project from one place. But consider this in relation to the next points.
  • Hash checking. If someone were to have a constraint file that said foo @ https://some/url and a requirements file that said foo --hash=xxx, what should happen? Should we hash-check the URL? Is it of any practical use? Should we select one of multiple URL constraints based on the hash?
  • Wheel compatibility tags. Should URLs for constraints be required to match the target wheel compatibility rules? Should we allow multiple URLs and pick the one that matches the target system? Have we just implemented a package index via constraint URLs?
  • Should we reject the edge cases that we don't support, or are we OK with just leaving them as "undefined behaviour - do not use"? In particular, even detecting some of these cases is costly.

There are quite likely other questions. These are basically just some that came up in the process of trying to see how to implement this and saying "ouch, that would be a problem" a few times 🙂

As I said, it's almost certainly possible to come up with a well defined spec for this feature. And maybe that spec would even match the current behaviour. But we don't have that spec right now, and no-one has really done any of the work needed to come up with one.

So I propose that we consider URLs as constraints as "deprecated pending work being done to properly define the feature"1, and assume that they won't be supported in at least the initial version of the new resolver. People currently using URLs in constraint files will need to either stick to the old resolver, or find a workaround.

The only actual user of URLs as constraints that I am aware of is @sbidoul, who explained his use here. Maybe there are (more or less clumsy) workarounds that could be used instead? @dhellmann seemed OK here with not including this in the new resolver, at least initially.

To reiterate, this is not a blanket refusal to accept this feature. It's simply noting that we need to clarify the design in order to rewrite it for the new resolver, and that we're not blocking the release of the new resolver on this feature.

1 This has horrible echoes of the dependency links mess. I really want to not have that happen again, so I want to be very clear that the plan this time is to drop the feature without any replacement if necessary, and not to leave it hanging around forever "until we have the replacement defined".

@sbidoul
Copy link
Member

@sbidoul sbidoul commented May 30, 2020

@pfmoore don't the edge case you mention also arise when direct URLs are used as regular install-requires?

@pfmoore
Copy link
Member

@pfmoore pfmoore commented May 30, 2020

@sbidoul I have no idea to be honest. I've not spotted any test failures that do that, but if there are any, then yes, that's something we'd need to address (and having addressed it, we may well then have a better chance of implementing URLs as constraints).

@uranusjr
Copy link
Member

@uranusjr uranusjr commented May 30, 2020

OTOH I very much doubt there are robust tests or user experience on this. The legacy resolver handles non-user-specified direct URLs very poorly, so there are likely many edge cases currently precluded by other restrictions. I’m +1 on delaying thoughts on this until we are able to make the new resolver widely available for a while, to let the users come up with crazy cool use cases that expose considerations we missed.

@pfmoore
Copy link
Member

@pfmoore pfmoore commented May 31, 2020

Agreed, this whole area is really emergent behaviour from a set of features that were implemented independently and never really considered in combination.

This is probably something that needs to be covered under #6536 (new resolver rollout) which is something that needs to be prioritised at some point. There will be areas where the new resolver works differently, that either aren't covered by tests, or where we've made a decision to keep a difference. and we need to look at how we get feedback on that.

@brainwane brainwane added this to Post-release work in New Resolver Implementation via automation Jun 24, 2020
brainwane added a commit to brainwane/pip that referenced this issue Jul 28, 2020
@sbidoul sbidoul added the C: direct url Direct URL references (PEP 440, PEP 508, PEP 610) label Aug 4, 2020
@thenewguy
Copy link

@thenewguy thenewguy commented Aug 14, 2020

I had opened an issue here #8757

Just wanted to add a mention here that it is very helpful to be able to provide the url for installation a package from as a constraint when waiting for a contributed patch to be applied by upstream and released.

I.e. something like https://github.com/some/repo/archive/bugfix-issue-foo.tar.gz#egg=somepackage to allow installing the bugfix for package "somepackage" until upstream can cut a new release. Otherwise all packages have to be vendored and renamed and it is very complicated and duplicates a lot of work.

@thenewguy
Copy link

@thenewguy thenewguy commented Aug 14, 2020

Right now if you drop https://github.com/some/repo/archive/bugfix-issue-foo.tar.gz#egg=somepackage in a constraint file and then install a requirements.txt that lists somepackage, you get https://github.com/some/repo/archive/bugfix-issue-foo.tar.gz. Please do not remove that workflow if you are able to keep it in some form

@pfmoore
Copy link
Member

@pfmoore pfmoore commented Aug 14, 2020

@thenewguy Thanks for your input. I've seen a few people describe use cases for URLs as constraints that all seem to have the same underlying motivation, and I'm fairly clear that we should have this feature in some form.

However, the key here is defining the details. I posted above a list of questions about how URLs as constraints should interact with other features of pip, and so far, no-one has come up with any answers. I don't personally have any intuition on what the answers should be, so it's essentially impossible to implement the feature for the new resolver. To be clear, the existing implementation is tightly linked to the old resolver code - it's not a case of "keeping what's there", we absolutely have to do a complete re-implementation, and as such, we need to know what to implement.

Please do not remove that workflow if you are able to keep it in some form

As I say, we aren't able to keep it. What we can do is to re-implement it. But to do that we need to understand what to implement - and "what the old implementation does" isn't even meaningful in terms of the new resolver...

@thenewguy
Copy link

@thenewguy thenewguy commented Aug 14, 2020

As I say, we aren't able to keep it. What we can do is to re-implement it. But to do that we need to understand what to implement - and "what the old implementation does" isn't even meaningful in terms of the new resolver...

Gotcha - well in my experience - this is typically used for temporary bugfix releases. It sounds like there is a lot more to consider than my knowledge of pip's internals allow, but imo, if a named package url is provided, it should be forced and installed regardless of any version specifier. If that is too much, perhaps failing if the version doesn't match. I've never used it in combination with strict versioning (because as a bugfix there really isn't a version to it as I am not in control of the release process for upstream). So perhaps, conceptually, it is an override rather than a constraint. I've not come across another way to do this.

However, it is also handy when working with a pre-release package that isn't published somewhere like pypi yet. Although this case is easier to work around than the previous use.

@dhellmann
Copy link

@dhellmann dhellmann commented Aug 14, 2020

I don't use the feature, but it sounds like maybe one interpretation of how others use it is to (partially?) disable the resolver for a specific dependency and use the given distribution as the only available location and version. Maybe that implies some other behaviors about how that affects the other dependencies, I'm not sure. For example, should the resolver assume that the distribution at the given URL meets the version requirements of anything that depends on it, even if perhaps the version number in the distribution metadata says otherwise?

That would be kind of a "just do what I tell you" mode, which given the conversation elsewhere about how to reduce the scope of pip itself and encourage an ecosystem of surrounding tools, might make a lot of these sorts of features buildable outside of pip. For example, if pip could report the results of resolving a requirement set by producing a list of URLs without installing them (maybe it can do this already, or it means building another tool on top of the resolver library?), then someone could build that full list from a partial requirements.txt, edit the results to replace the URL for the dependency they are patching, and then install from that list of URLs without resolving dependencies at all (with pip, or something else) to get exactly what they want. That would give the user the convenience of pip helping to build the full dependency list, and mean that PyPA developers wouldn't have to entertain every variation of dependency management that anyone can come up with.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Aug 15, 2020

The “do as I tell you” usage is more difficult to implement, and should be discussed separately IMO (#8076 covers it). Constraints are currently implemented as adding to existing requirements. This makes them fit nicer with the rest of dependency resolution logic, and relatively easier to make sense. This is enough for most use cases in practice as well, since existing dependencies are correct most of the time, and a URL constraint just needs to add additional information to tell the resolver to specifically use that one particular artifact, much alike how version constraints do.

@pfmoore
Copy link
Member

@pfmoore pfmoore commented Aug 15, 2020

if a named package url is provided, it should be forced and installed regardless of any version specifier

Conceptually, that behaviour doesn't feel like a "constraint" to me, but more like an "override". I think it's important that constraints do what the name suggests, for both discoverability and understandability reasons. Overrides are being discussed separately in #8076 as @uranusjr says.

For me, conceptually, a URL constraint says "you can only get this project from this specific URL. If what's there doesn't satisfy the dependencies pip has calculated, you're out of luck".

To be fair, that logic mostly answers my above questions:

  • Multiple URLs - not allowed, reject as an error.
  • Hash checking - we should check the hash and fail if it doesn't match.
  • Wheel compatibility tags - fail with an error if the URL points to an incompatible wheel.

For the last question, "should we reject the edge cases that we don't support" I would be pragmatic and say that if we end up in a code branch that the above logic doesn't give us an answer for, we fail with an error saying it's unsupported. On the other hand, we don't go out of our way to check for special cases, and we document the intention and explicitly state in the documentation that any other usage is not supported, and behaviour is undefined and may change or be removed without notice.

If that's an acceptable approach, we can go ahead on that basis. (However, note that the period of paid work on developing the new resolver has completed now, so implementing this will be done on volunteer time - personally, I'd like to look at it but I don't know when I'll next have time to do so).

@dhellmann
Copy link

@dhellmann dhellmann commented Aug 15, 2020

The “do as I tell you” usage is more difficult to implement, and should be discussed separately IMO (#8076 covers it).

Sure, I'll read that, too.

Constraints are currently implemented as adding to existing requirements. This makes them fit nicer with the rest of dependency resolution logic, and relatively easier to make sense. This is enough for most use cases in practice as well, since existing dependencies are correct most of the time, and a URL constraint just needs to add additional information to tell the resolver to specifically use that one particular artifact, much alike how version constraints do.

Sure, that's another way to look at it. I was trying to point out that if the phases of resolving the dependencies and installing the packages were exposed explicitly, then many (most?) other use cases, such as this one, could be addressed by modifying the output of the resolver before passing the list to the installer. Those modifications could be left up to individual users or authors of other tools that integrate with pip, but pip's resolver wouldn't have to contain the additional complexity.

@pfmoore
Copy link
Member

@pfmoore pfmoore commented Aug 15, 2020

I was trying to point out that if the phases of resolving the dependencies and installing the packages were exposed explicitly, then many (most?) other use cases, such as this one, could be addressed by modifying the output of the resolver before passing the list to the installer.

That's an interesting idea. We could have something like pip install --resolve-only --out=some_file.txt and pip install --from-resolve-data=some_file.txt. I could see some pretty significant issues to consider here (what if the pip options used in the two phases were different, for a start?) and I can't imagine that we'd ever support editing that intermediate file (it's expected that people do this, but it would have to be on a "we won't help if you break stuff" basis, I'd have thought), but I can see it would be useful.

(Of course, taking that idea to its logical conclusion, we'd break pip up into a suite of tools doing the various "bits" of the process, much like the Unix idea of combining many small tools - and I doubt we're actually likely to go down that route in reality).

@dhellmann
Copy link

@dhellmann dhellmann commented Aug 15, 2020

I was trying to point out that if the phases of resolving the dependencies and installing the packages were exposed explicitly, then many (most?) other use cases, such as this one, could be addressed by modifying the output of the resolver before passing the list to the installer.

That's an interesting idea. We could have something like pip install --resolve-only --out=some_file.txt and pip install --from-resolve-data=some_file.txt. I could see some pretty significant issues to consider here (what if the pip options used in the two phases were different, for a start?)

Yes, that's close to what I was thinking. It might be easier to break down which options apply to each phase by thinking about new sub-commands with names like pip resolve and pip deploy, with pip install encompassing both phases transparently. Most of the options to the existing install command would apply to the resolve command, but not to deploy. Separate sub-commands also has the benefit of separate argument parsers, and if the options aren't available for pip deploy, they can't be different than the values given to pip resolve. :-)

and I can't imagine that we'd ever support editing that intermediate file (it's expected that people do this, but it would have to be on a "we won't help if you break stuff" basis, I'd have thought), but I can see it would be useful.

Yes, exactly. The deploy step would just take the data as input and do what it needs to do so the listed packages were on the import path (downloading things, turning them into wheels, writing files to the filesystem, etc.). It would only work from the list, though, without applying any additional rules or processing. It has to assume that the list is "correct". It would be up to the user to make the list contain what they want, and if what they want is broken somehow then that's not pip's problem.

I originally said a list of URLs, but the output of the resolve phase might be easier to consume if it includes more of the data that the resolver has. For example, the original requirement, the reason for a dependency being added if it wasn't in the original requirement list, the URL to the package, an optional second URL for a local cached copy, etc. A lot of that data could be optional because deploy wouldn't need it, but a program that wanted to modify the data could use it to make decisions about the modifications.

(Of course, taking that idea to its logical conclusion, we'd break pip up into a suite of tools doing the various "bits" of the process, much like the Unix idea of combining many small tools - and I doubt we're actually likely to go down that route in reality).

Separate sub-commands may make it easier for users to understand conceptually, but I wouldn't go so far as to create separate executables or main programs. And I wouldn't necessarily say that pip install should write the intermediate data to a file before deploying, but internally it should build the same data structure with the resolver and pass it to the deployment code.

@pradyunsg
Copy link
Member Author

@pradyunsg pradyunsg commented Aug 15, 2020

IIUC, we're going in the direction of #53, lockfiles, #7819 here.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Dec 4, 2020

@mwchase I think you read the right code, and tour assessment is correct. I have not tried to figure out the details though, so maybe more edge cases would pop up once the implementation is written.

@mwchase
Copy link
Contributor

@mwchase mwchase commented Dec 6, 2020

I've figured out how most of the feature interactions I've thought about so far work out, but one of them is harder, because it involves code that doesn't exist yet.

Factory._make_candidate_from_link() has a TODO that says "Check already installed candidate, and use it if the link and editable flag match." Currently, the source_link field (which I think is what I want to key off of) is None for AlreadyInstalledCandidates, which I assume is what the method would return for the given case. My assumption/hope is that, in order to implement this functionality, the field would have to be populated when applicable, so checking against it would still work.

I assume there's also some subtlety around extras, though my initial thought/hope is, that since extras can't appear in constraints (modulo some proposed changes in other issues that wouldn't affect things), the constraint should apply to the base package and not touch the virtual one.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Dec 6, 2020

original_link is where pip should download data to install, so it should be set to None for AlreadyInstalledCandidate because pip does not need to download anything for it. The URL that an already-installed candidate was installed from is recorded in direct_url.json (PEP 610), and read with direct_url_helpers.dist_get_direct_url().

@mwchase
Copy link
Contributor

@mwchase mwchase commented Jan 21, 2021

I'm currently writing tests and taking notes on implementation details. Here's another question to consider: excluding URL constraints implicitly excludes editable constraints. If URL constraints are possible, should editable constraints be possible, or should they be explicitly excluded?

@pfmoore
Copy link
Member

@pfmoore pfmoore commented Jan 21, 2021

Editable constraints definitely should not be possible - "editable" is about how you install things, not about what you install. Constraints are entirely about what gets installed, so editable makes no sense there.

@mwchase
Copy link
Contributor

@mwchase mwchase commented Jan 21, 2021

Excellent, I didn't really want to plumb that through. So, editable constraints should be specifically disallowed after URLs are allowed, and whether a candidate is editable is unrelated to whether it satisfies the constraint.

EDIT: This does mean that, if an environment uses editable installations and does the pip freeze > constraints.txt from #8210 (comment), then there needs to be a processing step to remove leading -e. But at least in that situation, the installation will be possible.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Jan 22, 2021

EDIT: This does mean that, if an environment uses editable installations and does the pip freeze > constraints.txt […] there needs to be a processing step to remove leading -e. But at least in that situation, the installation will be possible.

Yes. The same issue is raised in #9209 as well, and I think it is not unreasonable for pip to ignore these invalid things with a warning instead of rage quit here. But this should be a separate feature request and handled on it own.

@mwchase
Copy link
Contributor

@mwchase mwchase commented Feb 15, 2021

I'm finally trying to implement this instead of writing tests and thinking really hard about the design, and I've run into another thing that didn't occur to me. It looks like the legacy resolver ignores fragments when comparing URLs. Is that desirable behavior? My current implementation does not, but that's not a conscious decision on my part.

@sbidoul
Copy link
Member

@sbidoul sbidoul commented Feb 15, 2021

The wheel cache currently only considers the subdirectory and hash fragments. So for consistency I would do the same when comparing URLs for this purpose.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Feb 17, 2021

The new resolver currently considers the whole Link object’s equality though, so maybe it’d be best to change the logic together.

@mwchase
Copy link
Contributor

@mwchase mwchase commented Feb 22, 2021

Okay, I've got an initial implementation. It's not yet ready for a PR; here's what needs to happen first:

  • Change the link comparison logic, if we don't want to match the current cache behavior, and update the affected tests.
  • Improve organization and style issues in my code changes.
  • News fragment
  • Documentation update (possibly change the documentation I linked upthread, and lay out the pip freeze > constraints.txt use case, with caveats)

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Feb 22, 2021

Changing Link’s equality would probably be problematic (nobody really knows what implications this has), so I’d write a helper def links_equivalent(link1, link2) somewhere (probably models/link.py) and just use that instead. (Remember to put a clear docstring to explain why it is needed and when it should be used instead of ==).

@mwchase
Copy link
Contributor

@mwchase mwchase commented Feb 22, 2021

What I currently have is some helper functions that approximate the current cache-equality. They currently live next to the Constraint class, because that's where they're used. I'm fine moving them, I just first want to check that I'm actually implementing cache-equality, then try to cut down on some duplication in my implementation.

@mwchase
Copy link
Contributor

@mwchase mwchase commented Feb 25, 2021

Actually, on further inspection, I'm not sure that the _get_cache_path_for_parts function is relevant for comparing links in resolution. It looks like it's basically an optimization around the fact that multiple URLs can point to the same file, in a way that can be determined entirely from the syntax of the URLs.

I would rather have the behavior of URL constraints be consistent with URL requirements in the new resolver, than with URL constraints in the old resolver. Is that reasonable?

Making these changes will require changes to code that I needed to clean up anyway, as well as putting slightly more effort into rewriting one test.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Feb 25, 2021

Actually, on further inspection, I'm not sure that the _get_cache_path_for_parts function is relevant for comparing links in resolution.

You’re correct, it is not. During resolution, links are compared using the built-in ==:

def __eq__(self, other):
# type: (Any) -> bool
if isinstance(other, self.__class__):
return self._link == other._link
return False

which in turn delegates logic to KeyBasedCompareMixin._compare:

def _compare(self, other, method):
# type: (Any, Callable[[Any, Any], bool]) -> bool
if not isinstance(other, self._defining_class):
return NotImplemented
return method(self._compare_key, other._compare_key)

which would compare KeyBasedCompareMixin._compare_key, which is identical to Link._url:

self._url = url
self.comes_from = comes_from
self.requires_python = requires_python if requires_python else None
self.yanked_reason = yanked_reason
super().__init__(key=url, defining_class=Link)
self.cache_link_parsing = cache_link_parsing

In other words, two links are considered identical during resolution only if their URLs match completely. This is the safest route, but is known to have caused issues to some people (#9429 (comment)) that may also apply to constraints, so I think we’ll need to revisit them together. So I would like the constraint implementation to also introduce a link comparison function to replace the == comparison used above.

Does that make sense?

@pradyunsg
Copy link
Member Author

@pradyunsg pradyunsg commented Feb 27, 2021

Umm... Why are we touching the legacy resolver here?

@mwchase
Copy link
Contributor

@mwchase mwchase commented Feb 27, 2021

So far as I know, I didn't.

mwchase added a commit to mwchase/pip that referenced this issue Feb 28, 2021
mwchase added a commit to mwchase/pip that referenced this issue Feb 28, 2021
sebageek added a commit to sapcc/networking-aci that referenced this issue Apr 20, 2021
We need to remove aci dependencies from the requirements, as with them
present we cannot run the tests and they are not published by Cisco.

We also need to pin the pip version, as at the time of this writing URL
dependencies are not supported. When they will be supported by a current
release this can be removed again. See pypa/pip#8253
sebageek added a commit to sapcc/networking-aci that referenced this issue Apr 21, 2021
We need to remove aci dependencies from the requirements, as with them
present we cannot run the tests and they are not published by Cisco.

We also need to pin the pip version, as at the time of this writing URL
dependencies are not supported. When they will be supported by a current
release this can be removed again. See pypa/pip#8253
sebageek added a commit to sapcc/networking-aci that referenced this issue Apr 21, 2021
We need to pin the pip version, as at the time of this writing URL
dependencies are not supported. When they will be supported by a current
release this can be removed again.

See pypa/pip#8253
sebageek added a commit to sapcc/networking-arista that referenced this issue Apr 21, 2021
Tox dependencies are updated for Ussuri.

We need to pin the pip version, as at the time of this writing URL
dependencies are not supported. When they will be supported by a current
release this can be removed again.

See pypa/pip#8253
inmantaci added a commit to inmanta/inmanta-core that referenced this issue Apr 27, 2021
Bumps [pip](https://github.com/pypa/pip) from 21.0.1 to 21.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>21.1 (2021-04-24)</h1>
<h2>Process</h2>
<ul>
<li>Start installation scheme migration from <code>distutils</code> to <code>sysconfig</code>. A
warning is implemented to detect differences between the two implementations to
encourage user reports, so we can avoid breakages before they happen.</li>
</ul>
<h2>Features</h2>
<ul>
<li>Add the ability for the new resolver to process URL constraints. (<code>[#8253](pypa/pip#8253) &lt;https://github.com/pypa/pip/issues/8253&gt;</code>_)</li>
<li>Add a feature <code>--use-feature=in-tree-build</code> to build local projects in-place
when installing. This is expected to become the default behavior in pip 21.3;
see <code>Installing from local packages &lt;https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages&gt;</code>_
for more information. (<code>[#9091](pypa/pip#9091) &lt;https://github.com/pypa/pip/issues/9091&gt;</code>_)</li>
<li>Bring back the &quot;(from versions: ...)&quot; message, that was shown on resolution failures. (<code>[#9139](pypa/pip#9139) &lt;https://github.com/pypa/pip/issues/9139&gt;</code>_)</li>
<li>Add support for editable installs for project with only setup.cfg files. (<code>[#9547](pypa/pip#9547) &lt;https://github.com/pypa/pip/issues/9547&gt;</code>_)</li>
<li>Improve performance when picking the best file from indexes during <code>pip install</code>. (<code>[#9748](pypa/pip#9748) &lt;https://github.com/pypa/pip/issues/9748&gt;</code>_)</li>
<li>Warn instead of erroring out when doing a PEP 517 build in presence of
<code>--build-option</code>. Warn when doing a PEP 517 build in presence of
<code>--global-option</code>. (<code>[#9774](pypa/pip#9774) &lt;https://github.com/pypa/pip/issues/9774&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fixed <code>--target</code> to work with <code>--editable</code> installs. (<code>[#4390](pypa/pip#4390) &lt;https://github.com/pypa/pip/issues/4390&gt;</code>_)</li>
<li>Add a warning, discouraging the usage of pip as root, outside a virtual environment. (<code>[#6409](pypa/pip#6409) &lt;https://github.com/pypa/pip/issues/6409&gt;</code>_)</li>
<li>Ignore <code>.dist-info</code> directories if the stem is not a valid Python distribution
name, so they don't show up in e.g. <code>pip freeze</code>. (<code>[#7269](pypa/pip#7269) &lt;https://github.com/pypa/pip/issues/7269&gt;</code>_)</li>
<li>Only query the keyring for URLs that actually trigger error 401.
This prevents an unnecessary keyring unlock prompt on every pip install
invocation (even with default index URL which is not password protected). (<code>[#8090](pypa/pip#8090) &lt;https://github.com/pypa/pip/issues/8090&gt;</code>_)</li>
<li>Prevent packages already-installed alongside with pip to be injected into an
isolated build environment during build-time dependency population. (<code>[#8214](pypa/pip#8214) &lt;https://github.com/pypa/pip/issues/8214&gt;</code>_)</li>
<li>Fix <code>pip freeze</code> permission denied error in order to display an understandable error message and offer solutions. (<code>[#8418](pypa/pip#8418) &lt;https://github.com/pypa/pip/issues/8418&gt;</code>_)</li>
<li>Correctly uninstall script files (from setuptools' <code>scripts</code> argument), when installed with <code>--user</code>. (<code>[#8733](pypa/pip#8733) &lt;https://github.com/pypa/pip/issues/8733&gt;</code>_)</li>
<li>New resolver: When a requirement is requested both via a direct URL
(<code>req @ URL</code>) and via version specifier with extras (<code>req[extra]</code>), the
resolver will now be able to use the URL to correctly resolve the requirement
with extras. (<code>[#8785](pypa/pip#8785) &lt;https://github.com/pypa/pip/issues/8785&gt;</code>_)</li>
<li>New resolver: Show relevant entries from user-supplied constraint files in the
error message to improve debuggability. (<code>[#9300](pypa/pip#9300) &lt;https://github.com/pypa/pip/issues/9300&gt;</code>_)</li>
<li>Avoid parsing version to make the version check more robust against lousily
debundled downstream distributions. (<code>[#9348](pypa/pip#9348) &lt;https://github.com/pypa/pip/issues/9348&gt;</code>_)</li>
<li><code>--user</code> is no longer suggested incorrectly when pip fails with a permission
error in a virtual environment. (<code>[#9409](pypa/pip#9409) &lt;https://github.com/pypa/pip/issues/9409&gt;</code>_)</li>
<li>Fix incorrect reporting on <code>Requires-Python</code> conflicts. (<code>[#9541](pypa/pip#9541) &lt;https://github.com/pypa/pip/issues/9541&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/2b2a268d25963727c2a1c805de8f0246b9cd63f6"><code>2b2a268</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/ea761a6575f37b90cf89035ee8be3808cf872184"><code>ea761a6</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/2edd3fdf2af2f09dce5085ef0eb54684b4f9bc04"><code>2edd3fd</code></a> Postpone a deprecation to 21.2</li>
<li><a href="https://github.com/pypa/pip/commit/3cccfbf169bd35133ee25d2543659b9c1e262f8c"><code>3cccfbf</code></a> Rename mislabeled news fragment</li>
<li><a href="https://github.com/pypa/pip/commit/21cd124b5d40b510295c201b9152a65ac3337a37"><code>21cd124</code></a> Fix NEWS.rst placeholder position</li>
<li><a href="https://github.com/pypa/pip/commit/e46bdda9711392fec0c45c1175bae6db847cb30b"><code>e46bdda</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9827">#9827</a> from pradyunsg/fix-git-improper-tag-handling</li>
<li><a href="https://github.com/pypa/pip/commit/0e4938d269815a5bf1dd8c16e851cb1199fc5317"><code>0e4938d</code></a> 📰</li>
<li><a href="https://github.com/pypa/pip/commit/ca832b2836e0bffa7cf95589acdcd71230f5834e"><code>ca832b2</code></a> Don't split git references on unicode separators</li>
<li><a href="https://github.com/pypa/pip/commit/1320bac4ff80d76b8fba2c8b4b4614a40fb9c6c3"><code>1320bac</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9814">#9814</a> from pradyunsg/revamp-ci-apr-2021-v2</li>
<li><a href="https://github.com/pypa/pip/commit/e9cc23ffd97cb6d66d32dc3ec27cf832524bb33d"><code>e9cc23f</code></a> Skip checks on PRs only</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/21.0.1...21.1">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=21.0.1&new-version=21.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

</details>
inmantaci added a commit to inmanta/inmanta-core that referenced this issue May 18, 2021
Bumps [pip](https://github.com/pypa/pip) from 21.0.1 to 21.1.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>21.1.1 (2021-04-30)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Temporarily set the new &quot;Value for ... does not match&quot; location warnings level
to <em>DEBUG</em>, to hide them from casual users. This prepares pip 21.1 for CPython
inclusion, while pip maintainers digest the first intake of location mismatch
issues for the <code>distutils</code>-<code>sysconfig</code> transition. (<code>[#9912](pypa/pip#9912) &lt;https://github.com/pypa/pip/issues/9912&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>This change fixes a bug on Python <!-- raw HTML omitted -->`_)</li>
<li>Fix compatibility between distutils and sysconfig when the project name is unknown outside of a virtual environment. (<code>[#9838](pypa/pip#9838) &lt;https://github.com/pypa/pip/issues/9838&gt;</code>_)</li>
<li>Fix Python 3.6 compatibility when a PEP 517 build requirement itself needs to be
built in an isolated environment. (<code>[#9878](pypa/pip#9878) &lt;https://github.com/pypa/pip/issues/9878&gt;</code>_)</li>
</ul>
<h1>21.1 (2021-04-24)</h1>
<h2>Process</h2>
<ul>
<li>Start installation scheme migration from <code>distutils</code> to <code>sysconfig</code>. A
warning is implemented to detect differences between the two implementations to
encourage user reports, so we can avoid breakages before they happen.</li>
</ul>
<h2>Features</h2>
<ul>
<li>Add the ability for the new resolver to process URL constraints. (<code>[#8253](pypa/pip#8253) &lt;https://github.com/pypa/pip/issues/8253&gt;</code>_)</li>
<li>Add a feature <code>--use-feature=in-tree-build</code> to build local projects in-place
when installing. This is expected to become the default behavior in pip 21.3;
see <code>Installing from local packages &lt;https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages&gt;</code>_
for more information. (<code>[#9091](pypa/pip#9091) &lt;https://github.com/pypa/pip/issues/9091&gt;</code>_)</li>
<li>Bring back the &quot;(from versions: ...)&quot; message, that was shown on resolution failures. (<code>[#9139](pypa/pip#9139) &lt;https://github.com/pypa/pip/issues/9139&gt;</code>_)</li>
<li>Add support for editable installs for project with only setup.cfg files. (<code>[#9547](pypa/pip#9547) &lt;https://github.com/pypa/pip/issues/9547&gt;</code>_)</li>
<li>Improve performance when picking the best file from indexes during <code>pip install</code>. (<code>[#9748](pypa/pip#9748) &lt;https://github.com/pypa/pip/issues/9748&gt;</code>_)</li>
<li>Warn instead of erroring out when doing a PEP 517 build in presence of
<code>--build-option</code>. Warn when doing a PEP 517 build in presence of
<code>--global-option</code>. (<code>[#9774](pypa/pip#9774) &lt;https://github.com/pypa/pip/issues/9774&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fixed <code>--target</code> to work with <code>--editable</code> installs. (<code>[#4390](pypa/pip#4390) &lt;https://github.com/pypa/pip/issues/4390&gt;</code>_)</li>
<li>Add a warning, discouraging the usage of pip as root, outside a virtual environment. (<code>[#6409](pypa/pip#6409) &lt;https://github.com/pypa/pip/issues/6409&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/c53d88c4c374523425f4db6bef949090764465c0"><code>c53d88c</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/4417e7f4bef2b2c70767c1dbfe72f82cc6b7b83f"><code>4417e7f</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/0c29bfe48e650c5a428b77eba4af7f067c019cc8"><code>0c29bfe</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9912">#9912</a> from uranusjr/sysconfig-remove-warning-for-python-re...</li>
<li><a href="https://github.com/pypa/pip/commit/f56ec327b92ebe836e63e07cb2449a20e09dec38"><code>f56ec32</code></a> Make location mismatch messages DEBUG level</li>
<li><a href="https://github.com/pypa/pip/commit/999b121402302a00b235a0d443f5661b69d6fd60"><code>999b121</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9883">#9883</a> from uranusjr/isolated-pip-py36-compat</li>
<li><a href="https://github.com/pypa/pip/commit/f88420319db450aefbed1500f04e31be46874aaf"><code>f884203</code></a> Fallback to self-invoke via directory on 3.6</li>
<li><a href="https://github.com/pypa/pip/commit/7a77484a492c8f1e1f5ef24eaf71a43df9ea47eb"><code>7a77484</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9835">#9835</a> from jamescurtin/9831-bugfix</li>
<li><a href="https://github.com/pypa/pip/commit/914bcc3dba88179f4e88ce90b63660474ba795cd"><code>914bcc3</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/9838">#9838</a> from uranusjr/sysconfig-header-with-none-project</li>
<li><a href="https://github.com/pypa/pip/commit/2a009a0b8a5d8d03117897f0f11f9c1dcf2a4b5a"><code>2a009a0</code></a> Better explanatory comment</li>
<li><a href="https://github.com/pypa/pip/commit/e7b1722efeaf4ff403142847ce1b52caedd5efcd"><code>e7b1722</code></a> Set dist_name to UNKNOWN when empty outside venv</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/21.0.1...21.1.1">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=21.0.1&new-version=21.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

</details>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 29, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C: constraint Dealing with "constraints" (the -c option) C: direct url Direct URL references (PEP 440, PEP 508, PEP 610) state: needs discussion This needs some more discussion type: feature request Request for a new feature type: maintenance Related to Development and Maintenance Processes type: question User question
Projects
No open projects
New Resolver Implementation
  
Post-release work
Development

No branches or pull requests

7 participants