install doesn't detect conflicts with already installed packages #2687

Open
rbtcollins opened this Issue Apr 15, 2015 · 11 comments

Projects

None yet

6 participants

@rbtcollins
Contributor

NB: I haven't created a reproducer for this, but I'm fairly sure its an issue :).

Given A with two versions 1.0 and 2.0 on PyPI
B install_requires A~=1.0
C install_requires A~=2.0

pip install B
pip install C

will complete without error, leaving B broken.

@qwcode
Contributor
qwcode commented Apr 15, 2015

yea, B will be broke.

@xavfernandez
Contributor

Related to #2492

@rbtcollins
Contributor

So decisions:
1)
pip install X which depends on Y==1.0
pip install Z which requires Y==$otherversion
Should that a) error or b) change X to a version that supports otherversion
I think b) because pip install X again would already automatically change Y, without needing -U.

  1. should there be an escape hatch to say 'don't consider already installed things', and if so what spelling to give it?
@qwcode
Contributor
qwcode commented Apr 23, 2015

I would say a) error. changing the version of X seems radical to me.

as for "don't consider already installed things", we have --ignore-installed... could it live under that?

@piotr-dobrogost

b) change X to a version that supports otherversion

What about packages that depend on X? I mean doesn't trying to fix version of Y forces us to make full dependency resolution? Where do we draw a line? I have a feeling that there are only two options; either there's very simple (dump?) dependency resolution like currently – take the first match or we are forced to make full dependency resolution. I can't image how some middle ground could look like.

changing the version of X seems radical to me.

It sure feels like this taking into account there's almost no dependency resolution in pip now.

@rbtcollins
Contributor

@qwcode we already change versions of things the user didn't specify. I"m confused why you'd find doing that for something that conflicts radical.

--ignore-installed - yes,that seems reasonable.

@piotr-dobrogost yes, this would work transitively. I'm working on this in the context of having full dependency resolution...

@dstufft
Member
dstufft commented Apr 24, 2015

I think it makes sense to change the versions of things even if they are already installed. It's a deviation of how pip has worked in the past -- generally we treated things as one shot installs while ignoring what was already installed (to some degree, besides conflict detection).

I think this is trickier then it appears at first, in a way this is instead changing it so that pip install, instead of just doing a one shot install, instead it adds a new desired item to the environment and then resolves it. I think this means that we'll need to track why items were installed and treat it in the dependency resolution as if all of the pip install commands were really all run together, so that the behavior of pip install foo<3 bar has the same result as pip install foo<3 && pip install bar (assuming no changes in the index between the two commands).

@rbtcollins
Contributor

So, we'll need a persistent store of the user supplied constraints. I'm happy to implement $whatever, but do you pip maintainers have any specific constraints on/around this? E.g. concurrency safe, place to store it, format to use?

Off hand I'm thinking: a directory in site-packages with a name that isn't a legitimate package containing one file per user-supplied constraint, with the content of the file being in requirements.txt format, updates via temp+rename on linux, with temp+delete+rename on windows.

@xavfernandez
Contributor

Maybe we could use https://www.python.org/dev/peps/pep-0376/#requested file to store the user constraints ?

@qwcode
Contributor
qwcode commented Apr 24, 2015

we already change versions of things the user didn't specify.
I"m confused why you'd find doing that for something that conflicts radical.

I'm ignoring what we currently do, since it's not we want.

my concern is changing a top-level requirement due to a sub-dependency in a different execution.

let's say I explicitly install "X==1", then later install "Y==1", which depends on "X==2".

I think that should fail, vs it automatically assuming, it should change my version of X.

@dstufft 's point about tracking the history of executions as requirements in the environment makes sense, and would alleviate my concern I think.

@ionelmc
Contributor
ionelmc commented Apr 28, 2015

Does it really make sense to store "user supplied constraints", as opposed to "package provided constraints" (from install_requires or whatever)? What do you do when the user changes their mind about whatever constraints he had in the past? Looks like a user experience quandary to me.

IMO, pip should only check against constraints of already installed packages when upgrading packages, not some past constraints provided at cmdline 1 year ago (that the user surely forgot, and would make for a really bad surprise factor).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment