Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Note issues with Provides-Dist and Obsoletes-Dist #513

Merged
merged 3 commits into from May 29, 2018

Conversation

@ncoghlan
Copy link
Member

commented May 27, 2018

The design of Provides-Dist and Obsoletes-Dist was lifted pretty
much directly from the design of Linux system package managers,
without accounting for the fact that Linux distro repos are collaboratively
maintained curated collections of packages with the ability to make
repository wide changes, while PyPI is an open publishing platform
with few practical restrictions other than size quotas and the default
first-come-first-served policy on claiming names.

pypa/packaging-problems#154

Note issues with Provides-Dist and Obsoletes-Dist
The design of Provides-Dist and Obsoletes-Dist was lifted pretty
much directly from the design of Linux system package managers,
without accounting for the fact that Linux distro repos are collaboratively
maintained curated collections of packages with the ability to make
repository wide changes, while PyPI is an open publishing platform
with few practical restrictions other than size quotas and the default
first-come-first-served policy on claiming names.

@ncoghlan ncoghlan requested review from dstufft and pganssle May 27, 2018

@pganssle

This comment has been minimized.

Copy link
Member

commented May 27, 2018

I think it's good to make this distinction, but as I mentioned in the other thread, I don't think that these sorts of things are nearly as likely to cause malicious problems as they seem. If anything the fact that they aren't enforced would cause more problems.

Until a long-term solution is clear, I'd probably go with a less prescriptive and more descriptive phrasing, something like, "Currently tools like pip do not make any use of this information, and you should be aware that the information is not validated or vetted in any way."

@ncoghlan

This comment has been minimized.

Copy link
Member Author

commented May 27, 2018

The fact pip will currently let one project overwrite another project's files seems completely orthogonal to this issue to me. The answer to that is to get pip to stop doing it unless you explicitly tell it that it's OK.

@ncoghlan

This comment has been minimized.

Copy link
Member Author

commented May 27, 2018

(Note: I do think Conflicts-Dist is a good idea to explicitly prevent co-installation, since that's a project level assertion that "Co-installation with this other project won't work". Still orthogonal to why the installation tools have never implemented support for Provides-Dist or Obsoletes-Dist in any meaningful way though, which is that in the absence of active curation of the repository as a whole, it turns out that there really isn't much that installers can safely do with the information)

@pfmoore

This comment has been minimized.

Copy link
Member

commented May 27, 2018

The fact pip will currently let one project overwrite another project's files seems completely orthogonal to this issue to me. The answer to that is to get pip to stop doing it unless you explicitly tell it that it's OK.

Is there a pip issue for this? It sounds to me like a reasonable feature request.

@pradyunsg

This comment has been minimized.

Copy link
Member

commented May 27, 2018

Is there a pip issue for this? It sounds to me like a reasonable feature request.

Yep -- pypa/pip#4625.

@pganssle

This comment has been minimized.

Copy link
Member

commented May 29, 2018

The fact pip will currently let one project overwrite another project's files seems completely orthogonal to this issue to me. The answer to that is to get pip to stop doing it unless you explicitly tell it that it's OK.

Well they're at least related by the fact that a worse form of abuse than is possible with Obsoletes-Dist is already possible, and it does not seem to be causing major problems at the moment. I think the jury is or should still be out on whether pip should do anything with Obsoletes-Dist (I argue in the other thread that it should be a special case of Conflicts-Dist), which is why I would argue for documenting that these are not currently used for anything rather than what they should be used for.

@ncoghlan

This comment has been minimized.

Copy link
Member Author

commented May 29, 2018

I have no idea whether or not anyone is using them successfully in a private use scenario - they're perfectly fine in the context of a Linux-style curated repository, which is why I've worded the PR the way I have. (Private curated repositories are also one of the main available mechanisms for folks to defend against undocumented conflicts)

@pganssle

This comment has been minimized.

Copy link
Member

commented May 29, 2018

@ncoghlan I don't have any objection to using them in private curated repositories, but I think "should only be used" is too strong, because I find the argument in favor of this is pretty weak. If Obsoletes-Dist were actually used to either uninstall conflicting packages or just block the installation of conflicting packages, I think that would be an improvement over the status quo. Even if the package-overwriting behavior of pip were fixed, I don't see it as any sort of security issue to interpret Obsoletes-Dist as a special case of Conflicts-Dist (I think this is basically how it is used in curated repositories anyway, though the uninstall-the-old-thing logic may be automatic).

I'd be happier replacing the stronger wording with, "These are not currently used by pip or PyPI, but have historically been included in case they would be useful for other package managers, particularly curated repositories."

Private use only fields -> Rarely used fields
Paul Ganssle made a good case that it's better to be
descriptive regarding Provides-Dist and Obsoletes-Dist
(it's a matter of objective fact that they're rarely used, and
installers ignore them completely), rather than attempting
to be retroactively prescriptive.
@ncoghlan

This comment has been minimized.

Copy link
Member Author

commented May 29, 2018

Ah, I see now - yeah, that makes sense.

eb8c260 switches from my original prescriptive wording to more matter-of-fact descriptive wording.

@pganssle

This comment has been minimized.

Copy link
Member

commented May 29, 2018

@ncoghlan Thanks for putting up with my persnickettiness!

@ncoghlan ncoghlan merged commit 6f5fae7 into master May 29, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@ankostis

This comment has been minimized.

Copy link

commented Aug 17, 2018

It's been referred in this issue and others the Conflicts-Dist metadata, but i cannot find anything about it.

What is it going to be working?
Would it be preferable over Obsoletes-Dist because it would be respected only if all conflicting packages declare it, and resolve thus the "non-curation" of PyPi?

[edit] I don't know if pypa/packaging-problems#154 is more relevant to my question.

@theacodes theacodes deleted the ncoghlan-note-private-use-only-fields branch Aug 17, 2018

@merwok merwok referenced this pull request Oct 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.