Add "outdated" pip command #235

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
@dgladkov
Contributor

dgladkov commented Mar 19, 2011

This command lists all installed (or only local) packages that can be updated. Basically it performs search for all installed package names in one XML-RPC request, filters only installed packages (there is no way to do exact match) and displays only those, which version differ from the latest version on PyPI.

I don't know why something like that wasn't implemented earlier - and maybe there is a reason for that - but I'm unaware of it:)

Also I added basic test for this command, but I haven't tested --local and --index options yet.

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez Mar 20, 2011

Contributor

I'm not convinced this should get its own command since it's really just a subset of what we've discussed an "info" command could show. E.g. pip info --outdated could show the outdated packages, while pip info shows all installed.

Contributor

jezdez commented Mar 20, 2011

I'm not convinced this should get its own command since it's really just a subset of what we've discussed an "info" command could show. E.g. pip info --outdated could show the outdated packages, while pip info shows all installed.

@dgladkov

This comment has been minimized.

Show comment Hide comment
@dgladkov

dgladkov Mar 20, 2011

Contributor

If pip info shows only installed packages, what's the difference between pip info and pip freeze? Perhaps pip info could display package description the same way pip search does, but I'm not sure this will be useful.

Could you please point me a link to the discussion about pip info if it's in public or describe here or in GitHub issue what you believe needs to be done?

Contributor

dgladkov commented Mar 20, 2011

If pip info shows only installed packages, what's the difference between pip info and pip freeze? Perhaps pip info could display package description the same way pip search does, but I'm not sure this will be useful.

Could you please point me a link to the discussion about pip info if it's in public or describe here or in GitHub issue what you believe needs to be done?

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez Mar 20, 2011

Contributor

The difference between info and freeze is its purpose. info shows information about a package, while freeze generates a requirements file listing.

Contributor

jezdez commented Mar 20, 2011

The difference between info and freeze is its purpose. info shows information about a package, while freeze generates a requirements file listing.

@carljm

This comment has been minimized.

Show comment Hide comment
@carljm

carljm Mar 22, 2011

Contributor

Hi dgladkov, thanks for this contribution! In general I agree with jezdez that "outdated" is quite specific functionality to deserve its own top-level command, and I'd rather see it as a flag to a general "show installed packages" command. I'm not sure about the name pip info - it seems so general that it could mean almost anything. I kind of like pip status. But that's just bikeshedding.

Re where that discussion happened previously, it was on some tickets I believe, though I can't find them at the moment.

The other thing I wonder about this patch is that it kind of makes the assumption that everything is listed on PyPI, even though that's not really true. I often use custom indexes, find-links, etc, and these things don't have an XMLRPC interface. A more flexible "outdated" command would not use XMLRPC, it would actually use PackageFinder to go out looking for updated versions in the same way "install" does, and would thus be able to handle options like --index-url and --find-links. Of course, this would be slower... and in the long run I think we (i.e. Python packaging infrastructure) should be moving away from link-scraping. But in the short run it feels strange to me to introduce new features that make inflexible and incorrect assumptions about how people use pip.

At the very least I think this patch should check > not != when comparing versions. I will quite commonly have a more recent pre-release or beta-release of something installed than the latest that is on PyPI, and it would be wrong to have those listed as "outdated" when they are in fact the opposite.

Contributor

carljm commented Mar 22, 2011

Hi dgladkov, thanks for this contribution! In general I agree with jezdez that "outdated" is quite specific functionality to deserve its own top-level command, and I'd rather see it as a flag to a general "show installed packages" command. I'm not sure about the name pip info - it seems so general that it could mean almost anything. I kind of like pip status. But that's just bikeshedding.

Re where that discussion happened previously, it was on some tickets I believe, though I can't find them at the moment.

The other thing I wonder about this patch is that it kind of makes the assumption that everything is listed on PyPI, even though that's not really true. I often use custom indexes, find-links, etc, and these things don't have an XMLRPC interface. A more flexible "outdated" command would not use XMLRPC, it would actually use PackageFinder to go out looking for updated versions in the same way "install" does, and would thus be able to handle options like --index-url and --find-links. Of course, this would be slower... and in the long run I think we (i.e. Python packaging infrastructure) should be moving away from link-scraping. But in the short run it feels strange to me to introduce new features that make inflexible and incorrect assumptions about how people use pip.

At the very least I think this patch should check > not != when comparing versions. I will quite commonly have a more recent pre-release or beta-release of something installed than the latest that is on PyPI, and it would be wrong to have those listed as "outdated" when they are in fact the opposite.

@xster

This comment has been minimized.

Show comment Hide comment
@xster

xster Mar 30, 2011

Hi,

As a user, I think that this functionality is quite urgent regardless of how it would be called.
As of the naming/commands, I would suggest that a 'list' and an 'info' command generally seems to work fine. Listing of outdated (and other filtered) packages can be a subset of a list command. Otherwise, macports and homebrew have a dedicated 'outdated' command as well.

xster commented Mar 30, 2011

Hi,

As a user, I think that this functionality is quite urgent regardless of how it would be called.
As of the naming/commands, I would suggest that a 'list' and an 'info' command generally seems to work fine. Listing of outdated (and other filtered) packages can be a subset of a list command. Otherwise, macports and homebrew have a dedicated 'outdated' command as well.

@douglas

This comment has been minimized.

Show comment Hide comment
@douglas

douglas Apr 25, 2011

Im with @xster here. Hope this code is merged soon.

douglas commented Apr 25, 2011

Im with @xster here. Hope this code is merged soon.

@Uznick

This comment has been minimized.

Show comment Hide comment
@Uznick

Uznick Jun 3, 2011

This functionality is very important. Will it be merged anytime soon? :)

And I vote for dedicated 'outdated' command.

Uznick commented Jun 3, 2011

This functionality is very important. Will it be merged anytime soon? :)

And I vote for dedicated 'outdated' command.

@douglas

This comment has been minimized.

Show comment Hide comment
@douglas

douglas Jul 7, 2011

Any news on this issue ?

douglas commented Jul 7, 2011

Any news on this issue ?

@Uznick

This comment has been minimized.

Show comment Hide comment
@Uznick

Uznick Jul 7, 2011

Nobody cares?

Uznick commented Jul 7, 2011

Nobody cares?

@carljm

This comment has been minimized.

Show comment Hide comment
@carljm

carljm Jul 7, 2011

Contributor

Hmm, I commented above on why this implementation doesn't seem like the right approach, given that not all packages come from PyPI - haven't seen any response to that yet.

Contributor

carljm commented Jul 7, 2011

Hmm, I commented above on why this implementation doesn't seem like the right approach, given that not all packages come from PyPI - haven't seen any response to that yet.

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez Jul 8, 2011

Contributor

Yeah, I think we can all agree that the feature is useful, only that the patch needs improvement as mentioned above.

Contributor

jezdez commented Jul 8, 2011

Yeah, I think we can all agree that the feature is useful, only that the patch needs improvement as mentioned above.

maraujop added a commit to maraujop/pip that referenced this pull request Jul 9, 2011

Updating pip outdated to use `PackageFinder` instead. Adding extra op…
…tions like '--index-url' or '--find-links'. Changing version comparison to use > not !=

All work done here is based on @carljm suggestions made in pull request #235, so that pip outdated gets merged into master asap.
@maraujop

This comment has been minimized.

Show comment Hide comment
@maraujop

maraujop Jul 9, 2011

Contributor

Hi Carl,

I finally managed to upgrade pip outdated by @dgladkov so that it uses PackageFinder. Former version from @dgladkov failed to run in my computer, because of some packages I had. This one works, which doesn't mean is bug free.

I'm using a "private" method to get the local version installed, as you can see in my comments. I'm willing to update anything necessary so that this gets merged into master.

I've been thinking, that instead of using a dictionary installations we could use a RequirementSet and outdated command could get a parameter like --upgrade which also upgrade packages. That would probably make having a separate command for this more understandable.

Cheers,
Miguel

Contributor

maraujop commented Jul 9, 2011

Hi Carl,

I finally managed to upgrade pip outdated by @dgladkov so that it uses PackageFinder. Former version from @dgladkov failed to run in my computer, because of some packages I had. This one works, which doesn't mean is bug free.

I'm using a "private" method to get the local version installed, as you can see in my comments. I'm willing to update anything necessary so that this gets merged into master.

I've been thinking, that instead of using a dictionary installations we could use a RequirementSet and outdated command could get a parameter like --upgrade which also upgrade packages. That would probably make having a separate command for this more understandable.

Cheers,
Miguel

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez Jul 9, 2011

Contributor

Just wanted to give a quick idea for the command -- let's have a pip list command that gets a bunch of flags:

  • --outdated Returns a list of requirements that are outdated as request in this ticket
  • --uptodate Returns a list of requirements that are up-to-date
  • --local Returns a list of non-editable requirements
  • --editable Returns a list of editable requirements
  • --freeze An additive flag that formats the output in requirements files format

etc.

With the last option we'd be able to deprecate the freeze command (or simply hide it from pip help and promoting the use of pip list --freeze instead).

Contributor

jezdez commented Jul 9, 2011

Just wanted to give a quick idea for the command -- let's have a pip list command that gets a bunch of flags:

  • --outdated Returns a list of requirements that are outdated as request in this ticket
  • --uptodate Returns a list of requirements that are up-to-date
  • --local Returns a list of non-editable requirements
  • --editable Returns a list of editable requirements
  • --freeze An additive flag that formats the output in requirements files format

etc.

With the last option we'd be able to deprecate the freeze command (or simply hide it from pip help and promoting the use of pip list --freeze instead).

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez Jul 9, 2011

Contributor

BTW, given the fact that the other commands are imperatives, pip list fits better than pip status, IMO.

Contributor

jezdez commented Jul 9, 2011

BTW, given the fact that the other commands are imperatives, pip list fits better than pip status, IMO.

@maraujop

This comment has been minimized.

Show comment Hide comment
@maraujop

maraujop Jul 13, 2011

Contributor

Hi,

As I said in the IRC channel. I'm in favor of a pip list command and I can work on it. But before moving pip outdated to pip list --outdated I would love that a core dev reviews my patch.

I have two concerns about it:

  • Performance: PackageFinder.find_requirement is slow, there should be a way to create a batch of queries, to speed things up or maybe using threads.
  • That method returns a Link, that I think is always pointing to a PyPI url, right? So how should I proceed with packages installed from version control repositories.

Here is the last version of pip outdated:
https://github.com/maraujop/pip/blob/develop/pip/commands/outdated.py

Thanks, regards
Miguel

Contributor

maraujop commented Jul 13, 2011

Hi,

As I said in the IRC channel. I'm in favor of a pip list command and I can work on it. But before moving pip outdated to pip list --outdated I would love that a core dev reviews my patch.

I have two concerns about it:

  • Performance: PackageFinder.find_requirement is slow, there should be a way to create a batch of queries, to speed things up or maybe using threads.
  • That method returns a Link, that I think is always pointing to a PyPI url, right? So how should I proceed with packages installed from version control repositories.

Here is the last version of pip outdated:
https://github.com/maraujop/pip/blob/develop/pip/commands/outdated.py

Thanks, regards
Miguel

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez Jul 27, 2011

Contributor

@maraujop Thanks for the work, regarding your questions:

  • Yeah, extending PackageFinder.find_requirement or even replacing it with a saner implementation seems like a good idea. The distutils2 project might have a better implementation of the requirement discovery, not sure though.
  • Editable requirements (from VCS) shouldn't be covered by the --outdated flag for now (due to the fact that defining "outdated" for a specific VCS checkout is impossible).
Contributor

jezdez commented Jul 27, 2011

@maraujop Thanks for the work, regarding your questions:

  • Yeah, extending PackageFinder.find_requirement or even replacing it with a saner implementation seems like a good idea. The distutils2 project might have a better implementation of the requirement discovery, not sure though.
  • Editable requirements (from VCS) shouldn't be covered by the --outdated flag for now (due to the fact that defining "outdated" for a specific VCS checkout is impossible).
@carljm

This comment has been minimized.

Show comment Hide comment
@carljm

carljm Oct 23, 2011

Contributor

Hi Miguel - your code looks good to me in general. I know find_requirement might be slow, but I think we're better off doing this in stages; I don't want to hold up the feature for some kind of massive performance optimization or reorganization of find_requirement.

I agree with @jezdez that the --outdated flag should not return editable requirements. And I also agree that list is a good command name, and I like the proposed flags for it (although for consistency with other places, --local should mean "exclude packages outside the current virtualenv", not "non-editable packages").

Contributor

carljm commented Oct 23, 2011

Hi Miguel - your code looks good to me in general. I know find_requirement might be slow, but I think we're better off doing this in stages; I don't want to hold up the feature for some kind of massive performance optimization or reorganization of find_requirement.

I agree with @jezdez that the --outdated flag should not return editable requirements. And I also agree that list is a good command name, and I like the proposed flags for it (although for consistency with other places, --local should mean "exclude packages outside the current virtualenv", not "non-editable packages").

@ogirardot

This comment has been minimized.

Show comment Hide comment
@ogirardot

ogirardot Dec 10, 2011

Contributor

btw just for the record the yolk tool has this kind of feature with

yolk -U, --show-updates  Check PyPI for updates on package(s).
Contributor

ogirardot commented Dec 10, 2011

btw just for the record the yolk tool has this kind of feature with

yolk -U, --show-updates  Check PyPI for updates on package(s).
@pnasrat

This comment has been minimized.

Show comment Hide comment
@pnasrat

pnasrat May 13, 2012

Contributor

@maraujop have you had time to address the concerns of @jezdez and @carljm re --outdated and flag consistency?

Contributor

pnasrat commented May 13, 2012

@maraujop have you had time to address the concerns of @jezdez and @carljm re --outdated and flag consistency?

@lepture

This comment has been minimized.

Show comment Hide comment
@lepture

lepture May 28, 2012

When should this be merged ? If it will not be merged, please close it.

It has been one year since this pull request. But this feature is still not included in pip.

😭

lepture commented May 28, 2012

When should this be merged ? If it will not be merged, please close it.

It has been one year since this pull request. But this feature is still not included in pip.

😭

@jezdez

This comment has been minimized.

Show comment Hide comment
@jezdez

jezdez May 29, 2012

Contributor

@lepture It'll be merged when the code is ready, which takes a while sometimes.

Contributor

jezdez commented May 29, 2012

@lepture It'll be merged when the code is ready, which takes a while sometimes.

@pnasrat

This comment has been minimized.

Show comment Hide comment
@pnasrat

pnasrat Jul 1, 2012

Contributor

@maraujop any updates?

Contributor

pnasrat commented Jul 1, 2012

@maraujop any updates?

@xster

This comment has been minimized.

Show comment Hide comment
@xster

xster Jul 8, 2012

it's been over a year... let's put it in :)

xster commented Jul 8, 2012

it's been over a year... let's put it in :)

@maraujop

This comment has been minimized.

Show comment Hide comment
@maraujop

maraujop Jul 9, 2012

Contributor

Sorry, to take so long to answer. I planned back in time to address to move the code to a list command, but for some reason or other I didn't tackle it, although it shouldn't be much work. I would love to finish it, but I'm currently overwhelmed with work and organizing some Python events in Spain, so I'm not sure when I will have the time, sorry for the inconvenience.

Cheers,
Miguel

Contributor

maraujop commented Jul 9, 2012

Sorry, to take so long to answer. I planned back in time to address to move the code to a list command, but for some reason or other I didn't tackle it, although it shouldn't be much work. I would love to finish it, but I'm currently overwhelmed with work and organizing some Python events in Spain, so I'm not sure when I will have the time, sorry for the inconvenience.

Cheers,
Miguel

rafaelcaricio added a commit to rafaelcaricio/pip that referenced this pull request Oct 2, 2012

Updating pip outdated to use `PackageFinder` instead. Adding extra op…
…tions like '--index-url' or '--find-links'. Changing version comparison to use > not !=

All work done here is based on @carljm suggestions made in pull request #235, so that pip outdated gets merged into master asap.
@qwcode

This comment has been minimized.

Show comment Hide comment
@qwcode

qwcode Oct 3, 2012

Contributor

this being handled by pull #675

Contributor

qwcode commented Oct 3, 2012

this being handled by pull #675

@qwcode qwcode closed this Oct 3, 2012

gvalkov added a commit to gvalkov/pip that referenced this pull request Dec 9, 2012

Updating pip outdated to use `PackageFinder` instead. Adding extra op…
…tions like '--index-url' or '--find-links'. Changing version comparison to use > not !=

All work done here is based on @carljm suggestions made in pull request #235, so that pip outdated gets merged into master asap.

rafaelcaricio added a commit to rafaelcaricio/pip that referenced this pull request Dec 9, 2012

Updating pip outdated to use `PackageFinder` instead. Adding extra op…
…tions like '--index-url' or '--find-links'. Changing version comparison to use > not !=

All work done here is based on @carljm suggestions made in pull request #235, so that pip outdated gets merged into master asap.

qwcode added a commit to qwcode/pip that referenced this pull request Dec 18, 2012

Updating pip outdated to use `PackageFinder` instead. Adding extra op…
…tions like '--index-url' or '--find-links'. Changing version comparison to use > not !=

All work done here is based on @carljm suggestions made in pull request #235, so that pip outdated gets merged into master asap.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment