Optionally warn if `update` cannot install latest version of a package

[fingolfin]

The following has bitten me and colleagues many times: we want to install the latest version of a package (e.g. CxxWrap), so I do ] update CxxWrap, and I see this:

(v1.3) pkg> update CxxWrap
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
  Updating `~/.julia/environments/v1.3/Project.toml`
 [no changes]
  Updating `~/.julia/environments/v1.3/Manifest.toml`
 [no changes]

Odd, I wonder, nothing happened... Did I already have the latest version? Hm, let's check:

(v1.3) pkg> status CxxWrap
    Status `~/.julia/environments/v1.3/Project.toml`
  [b99e7846] BinaryProvider v0.5.8
  [1f15a43c] CxxWrap v0.8.2

Oh no, that's not the latest. Which leads me to my first question:

1. Is there any way to find out what the latest version of a package from the Julia REPL? Or get a list of all available package versions?

What's going on? Well, after some searching around, I may discover that some other package has a dependency on CxxWrap with an upper bound that prevents any newer version of CxxWrap installed. The only way I found to do that was to manually go through each installed/active package, and look at its dependencies. So my second question:

2. Is there any way to get a list of active packages that depend on a given package?

And finally, the feature request from the headline of this issue:

3. Would it be possible/acceptable/... to change the behavior of update to print a warning if a package is not updated to the latest due to a constraint?

I can simulate this by removing the package and then adding it back with a version constraint:

(v1.3) pkg> rm CxxWrap
  Updating `~/.julia/environments/v1.3/Project.toml`
  [1f15a43c] - CxxWrap v0.8.2
  Updating `~/.julia/environments/v1.3/Manifest.toml`
 [no changes]

(v1.3) pkg> add CxxWrap@0.9.0
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Singular [bcd08a7b]:
 Singular [bcd08a7b] log:
 ├─possible versions are: 0.1.0 or uninstalled
 ├─restricted to versions 0.1.0 by an explicit requirement, leaving only versions 0.1.0
 └─restricted by compatibility requirements with CxxWrap [1f15a43c] to versions: uninstalled — no versions left
   └─CxxWrap [1f15a43c] log:
     ├─possible versions are: [0.7.0-0.7.4, 0.8.0-0.8.2, 0.9.0] or uninstalled
     └─restricted to versions 0.9.0 by an explicit requirement, leaving only versions 0.9.0

But that requires me to know that a newer version is available (see question 1); I guess if there was a "pseudo version" to indicate the latest, that'd also work; something like add CxxWrape@latest ?

Also, it seems I have to remove the package first? At least I couldn't figure out a way to make a version specifier work with update; doing ]update CxxWrap@0.9.0 does the same as update CxxWrap.

CC @fieker

[KristofferC]

Yeah, providing some short info about this in the default Pkg.status as well as giving a hint to another command, say, Pkg.compat_status() that shows more extensively what is holding what back, makes sense to me.

[getzze]

Same problem here, It is very hard to know the last version of the packages and the dependency tree. So it is hard to know (1) that there is a problem, (2) how to solve it. I would like to tell what I did so it can help to identify the problems.

I was getting this error:

┌ Info: Precompiling DifferentialEquations [0c46a032-eb83-5123-abaf-570d42b7fbaa]
└ @ Base loading.jl:1273
WARNING: Method definition vec(Number) in module DiffEqDiffTools at ~/.julia/packages/DiffEqDiffTools/3mm8U/src/jacobians.jl:114 overwritten in module FiniteDiff at ~/.julia/packages/FiniteDiff/IPtYL/src/jacobians.jl:128.
  ** incremental compilation may be fatally broken for this module **

DiffEqDiffTools.jl has just been renamed to FiniteDiff.jl, so I ran ]up but everything seemed up to date...

So I identified the packages that still had a dependency to DiffEqDiffTools in the Manifest:

grep -B1 -A4 -i -n "DiffEqDiffTools" ~/.julia/environments/v1.3/Manifest.toml

That gave me DiffEqBase, OrdinaryDiffEq and StochasticDiffEq. I checked on their github page what was their last versions and they were not those I had in my Manifest file. So the next step was to run ]preview add [pkg]#master to force them to the last version and see which package was preventing them from updating. Here is the full sequence I ran on the REPL:

https://gist.github.com/getzze/4c76f701adb961d2ca99cc68297f84f7

I think it would be good to display if a package is not in its last version and also to have a function to list the dependency of a package, or even a dependency tree.

[KristofferC]

If your code requires a specific version you should generally put that in your Project file under [compat]. So if you require DifferentialEquations 0.8 you put

[compat]
DifferentialEquations = "0.8"

If that version cannot be installed, an error message with "the dependency of a package, or even a dependency tree." will be printed. If you have no compat entry you are saying that you are fine with whatever version the resolver can get you.

[getzze]

I am using the packages in rolling release :) Is there a way to specify that you always want the last version then?

[fieker]

On Fri, Feb 07, 2020 at 05:50:59AM -0800, Kristoffer Carlsson wrote: If your code requires a specific version you should generally put that in your Project file under `[compat]`. So if you require DifferentialEquations 0.8 you put ``` [compat] DifferentialEquations = "0.8" ``` If, that cannot be installed, an error message with "the dependency of a package, or even a dependency tree." will be printed. If you have no `compat` entry you are saying that you are fine with whatever version the resolver can get you.

Yes, this is fine. However I cannot figure out WHY the resolver is doing (or not doing) what it does, decisions are based on information that I usually do not have, namely dependencies of packages that I did not install, they are dependencies of s.th. else. It's the information about availability of (newer) versions together with the reason(s) why they are rejected that is missing.

…

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1655 (comment)

[BioTurboNick]

Just going to +1 this idea. Spent a while wondering why the dependency wasn't updating before I remembered to check my package's compat listing.

In case the issue is slightly different: My [compat] restricted a package to 0.5, but ]update did not warn me that v0.6 was available but being ignored due to [compat].

[azev77]

+1 from me as well, as mentioned in closed issue (1774) & Discourse.
This is one of my biggest usability problems.

[CarloLucibello]

Is there any workaround? i.e. any way to know which of the installed packages is not on the latest version and why?

[azev77]

@CarloLucibello
in the Julia package manager type status & it will show which version of a pkg you currently have installed.
Then go to the packages github page (eg MLJ.jl) and see if the version you have installed is the latest version.

[drcxcruz]

Julia definitely needs to automatically provide warning when a package cannot be updated with the ]update command. I would guess 100s if not 1000s of people do not have the latest version of their packages without even know it.