Check for latest version before running with pipx run

[anthem-bmk]

How would this feature be useful?

When using pipx run someapp, a version of someapp is cached for 14 days. There could be new releases within that timeframe that could be useful to have.

Describe the solution you'd like

I think my ideal solution would be to just automatically check for a newer version and upgrade if it exists. If you're already on the latest version, use the cache.

The next best solution (which would avoid any backwards compatibility issues) would be pipx run --latest someapp, that would work the same way but require an explicit flag.

A third solution, that might work well with the second, is having some sort of config you can set (e.g. ~/.pipxrc) that lets you say you always want the --latest flag.

Describe alternatives you've considered

The only real workaround I've found is to run the app a second time without cache, then a third time with the cache. This also only works if you know the version has changed and want to force it to change. For example:

pipx run someapp --version # v0.1.0
# some new update comes out
pipx run --no-cache someapp --version # v0.1.1
pipx run someapp --version # v0.1.1

Some downsides:

• If the app has a lot of dependencies, running --no-cache every time will be pretty inefficient.
• If you don't always run with --no-cache, you need to know somehow if there's a version update
• If you want to update once with --no-cache, it will delete the cache the first time and download the latest version, then the second time it'll download it again but actually cache it.

[cs01]

I like this idea. 14 days is kind of a long time. I tried to land on a good balance between checking for updates too often (and slowing things down), and not often enough (and getting stuck on old versions).

My main concern with the --latest flag is that it might take a long time to determine the version since things will have to be downloaded/cloned. @itsayellow you have done some work in this space. Is there a way to determine the latest version of a package quickly for something like this?

[itsayellow]

The easy way to do this is to parse pip search <package_name>. We'd then have to parse the output for the INSTALLED version and (latest) version. This would take as long as a pip search takes, which from an anecdotal test just now on my machine took 0.6 seconds.

Another way would be to do a query of pypi directly using their simple API. I don't know if this is any better than using pip, and it would be a bunch more work.

I couldn't figure out a way with pip (or otherwise) to just directly check a package's current version other than the above, but I may be missing something.

[anthem-bmk]

Yeah I'm not sure of any other way besides pip or the API. Looks like it could add 1s to execution, which isn't ideal by default but is probably worth it behind a flag if the primary goal is to always be up to date. Here are some crude examples of using pip vs the API:

$ time pip3 search cowsay | grep '^cowsay\ (' | grep -Eo '[0-9.]+'
2.0.3

real    0m1.106s
user    0m0.374s
sys     0m0.117s

$ time curl -s https://pypi.org/pypi/cowsay/json | json info.version
2.0.3

real    0m0.591s
user    0m0.053s
sys     0m0.050s

(that's using npm i -g json, but there are similar tools out there like jq)

Looks like the API is a little bit faster and cleaner to check (those grep commands could probably be simplified, but still don't need regex using json).

[itsayellow]

I didn't know how easy the api was to use, so based on @anthem-bmk's recipe above I tried it on my own computer and it returned in 0.01 seconds! I am blessed with a fast internet connection so it may be that the time for this is dominated by whatever internet the user has.

[itsayellow]

It looks to me like the most direct and fast implementation of this would use the json API of pypi, or the simple API.

Does anybody know if other indexes need to have the json API available, or the simple API available? This might be an issue for folks that specify --index-url or --extra-index-url or equivalent settings.

[cs01]

One second is definitely fine with me. My concern is for situations where we're installing from source or a vcs url. How would we handle that?

[anthem-bmk]

oh, that's a good point... pip search --index $JFROG_ARTIFACTORY_URL my_private_package works but as far as I can tell Artifactory doesn't have a compatible API.

It would make the code more complex, but maybe use pip search if an index is provided and use the API otherwise? That 0.01 seconds sounds pretty nice!

As a side note, I'm going down the rabbit hole of github issues trying to figure out the differences between --index and --index-url/--extra-index-url. It seems pip search uses --index while the others are used with pip install and there are some (not-so-?)subtle difference. I have a feeling pip search may not work very well for people using --index-url or --extra-index-url, though maybe it's configurable in pip.conf.

[anthem-bmk]

Another good point - maybe the flag only deals with packages that can be searched. A URL for example it could just assume is always exact and you would need to use --no-cache to refetch it. I think yet another flag like --invalidate-cache could be really handy (either in addition to or instead of --latest) that would remove the current cache but cache the new installation.

Does a relative url like ../asdf currently stay cached for 14 days as well? Maybe something like that could check setup.py (or equivalent)...? This sounds like it could get pretty hairy.

[uranusjr]

The simplest way to do this is probably to access the Simple API directly. There are clients like mousebender for that.

[anthem-bmk]

It looks like Artifactory does actually support the pypi Simple API, but not JSON. So if we went Simple all the way that could work... not sure how many other projects there are out there to support private repos and whether they also support the API. Looks like it's XML so it's not as easy to parse as JSON, but like @uranusjr said there are clients out there for that.

[uranusjr]

The Simple API is HTML (PEP 503), and pip uses it as the primary package discovery API, so it’s extremely likely to be supported by any given repository implementation.

[gaborbernat]

PR welcome.