New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-downloadable/installed dependencies shouldn't fail upgrades #2502

Closed
qwcode opened this Issue Mar 7, 2015 · 12 comments

Comments

Projects
None yet
4 participants
@qwcode
Contributor

qwcode commented Mar 7, 2015

if some pkgb depends on pkga (i.e. has it specified in install_requires), an upgrade of pkgb shouldn't fail if a download isn't found for pkga (e.g. if it's not in PyPI), if pkga is installed locally.

to recreate, create pkga and pkgb in local directories with setup.py's like so

pkga/setup.py

from setuptools import setup
setup(name='pkga',
      version='0.1')

pkgb/setup.py

from setuptools import setup
setup(name='pkgb',
      version='0.1',
      install_requires=['pkga'])

and run

> pip install -e pkga/
> pip install -U -e pkgb/
Could not find any downloads that satisfy the requirement pkga in ./pkga (from pkgb==0.6.1)
Collecting pkga (from pkgb==0.6.1)
  No distributions at all found for pkga in ./pkga (from pkgb==0.6.1)
@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Mar 7, 2015

Contributor

issue came out of discussion in #2472

Contributor

qwcode commented Mar 7, 2015

issue came out of discussion in #2472

@msabramo

This comment has been minimized.

Show comment
Hide comment
@msabramo

msabramo Mar 8, 2015

Contributor

I feel like there are 2 possible approaches to upgrading.

  1. Download/do the minimal amount required to satisfy the constraints. Be lazy and don't hit the network unless necessary. This is better for speed and greater ability to work offline, but you end up with older versions of packages and maybe more variation in people's environments, which could lead to people having problems that are fixed in later versions.
  2. Try to always upgrade to the latest. This tends to get people to the latest and to have more similar envs to each other and so maybe less bugs because folks are more likely to be on the same page. It does cause more downloads though so it makes pip slower and less likely to work when offline.

I think pip largely implements Approach 2.

I wonder if it would be useful to have pip support changing the strategy to Approach 1. But the fear I have is that this would make the code more complex, especially if no refactoring is done before to isolate the strategy from everything else.

Maybe it's worth documenting pip's overall strategy (if it isn't already) so we can at least point people to a URL to explain why they're seeing the behavior they're seeing?

Contributor

msabramo commented Mar 8, 2015

I feel like there are 2 possible approaches to upgrading.

  1. Download/do the minimal amount required to satisfy the constraints. Be lazy and don't hit the network unless necessary. This is better for speed and greater ability to work offline, but you end up with older versions of packages and maybe more variation in people's environments, which could lead to people having problems that are fixed in later versions.
  2. Try to always upgrade to the latest. This tends to get people to the latest and to have more similar envs to each other and so maybe less bugs because folks are more likely to be on the same page. It does cause more downloads though so it makes pip slower and less likely to work when offline.

I think pip largely implements Approach 2.

I wonder if it would be useful to have pip support changing the strategy to Approach 1. But the fear I have is that this would make the code more complex, especially if no refactoring is done before to isolate the strategy from everything else.

Maybe it's worth documenting pip's overall strategy (if it isn't already) so we can at least point people to a URL to explain why they're seeing the behavior they're seeing?

@remram44

This comment has been minimized.

Show comment
Hide comment
@remram44

remram44 Mar 8, 2015

Contributor

I'm very OK with 2) so long as failing to contact or find pkg on PyPI for recursive deps is not a failure.

Contributor

remram44 commented Mar 8, 2015

I'm very OK with 2) so long as failing to contact or find pkg on PyPI for recursive deps is not a failure.

@msabramo

This comment has been minimized.

Show comment
Hide comment
@msabramo

msabramo Mar 8, 2015

Contributor

@remram44: Apologies if this already came up - I'm on a phone so it's not so easy to browse around and check stuff...

I wonder if it would help if you added --find-links .. Maybe that would encourage pip to find pkga locally, though it might still try PyPI. I guess you could also try something like --find-index . --no-index an you might want to put those in a requirements.txt to make things easier.

See https://pip.pypa.io/en/latest/user_guide.html#fast-local-installs

Contributor

msabramo commented Mar 8, 2015

@remram44: Apologies if this already came up - I'm on a phone so it's not so easy to browse around and check stuff...

I wonder if it would help if you added --find-links .. Maybe that would encourage pip to find pkga locally, though it might still try PyPI. I guess you could also try something like --find-index . --no-index an you might want to put those in a requirements.txt to make things easier.

See https://pip.pypa.io/en/latest/user_guide.html#fast-local-installs

@msabramo

This comment has been minimized.

Show comment
Hide comment
@msabramo

msabramo Mar 8, 2015

Contributor

Ok, the 2 approaches I meant have been thoroughly documented already in the link I mentioned and in #59

Contributor

msabramo commented Mar 8, 2015

Ok, the 2 approaches I meant have been thoroughly documented already in the link I mentioned and in #59

@msabramo

This comment has been minimized.

Show comment
Hide comment
@msabramo

msabramo Mar 8, 2015

Contributor

@remram44 : Would #59 help you?

Contributor

msabramo commented Mar 8, 2015

@remram44 : Would #59 help you?

@remram44

This comment has been minimized.

Show comment
Hide comment
@remram44

remram44 Mar 8, 2015

Contributor

#59 would make the current upgrade be upgrade-all, which would probably still fail if some packages in the dependency tree are not on PyPI. Which would still be wrong. #59 unrelated IMHO.

Contributor

remram44 commented Mar 8, 2015

#59 would make the current upgrade be upgrade-all, which would probably still fail if some packages in the dependency tree are not on PyPI. Which would still be wrong. #59 unrelated IMHO.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Mar 10, 2015

Contributor

@msabramo I think pip has to do #2, but that doesn't mean it should suffer from the bug in the description. pip should just know that not finding any downloads is ok for an installed dependency, as long as it fulfills the requirement.

Contributor

qwcode commented Mar 10, 2015

@msabramo I think pip has to do #2, but that doesn't mean it should suffer from the bug in the description. pip should just know that not finding any downloads is ok for an installed dependency, as long as it fulfills the requirement.

@msabramo

This comment has been minimized.

Show comment
Hide comment
@msabramo

msabramo Mar 10, 2015

Contributor

@qwcode: I agree that not finding any downloads for a dependency shouldn't fail the whole upgrade.

Contributor

msabramo commented Mar 10, 2015

@qwcode: I agree that not finding any downloads for a dependency shouldn't fail the whole upgrade.

xavfernandez added a commit to xavfernandez/pip that referenced this issue Mar 13, 2015

remove Error when no versions are found
rely on if not applicable_versions test instead
fixes pypa#2502

xavfernandez added a commit to xavfernandez/pip that referenced this issue Mar 15, 2015

remove DistributionNotFound when no version is found
- and rely on if not applicable_versions test instead
- fixes pypa#2502
- rephrase the DistributionNotFound error message to fix other tests

@qwcode qwcode closed this in #2538 Mar 15, 2015

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Mar 15, 2015

Contributor

fixed in PR #2538

Contributor

qwcode commented Mar 15, 2015

fixed in PR #2538

qwcode added a commit that referenced this issue Mar 15, 2015

@piotr-dobrogost

This comment has been minimized.

Show comment
Hide comment
@piotr-dobrogost

piotr-dobrogost Mar 15, 2015

Does it work for every level of dependency tree or only for top level?

piotr-dobrogost commented Mar 15, 2015

Does it work for every level of dependency tree or only for top level?

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Mar 15, 2015

Contributor

you mean the PR fix? should be any level

Contributor

qwcode commented Mar 15, 2015

you mean the PR fix? should be any level

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment