]up only downgrades packages

[kescobo]

As mentioned on Slack, I have a weird situation where ]up is doing nothing but downgrading packages, though it doesn't seem like there's any incompatibility problems:

(ECHOAnalysis) pkg> up
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
Updating `~/repos/scratch/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
Updating `~/repos/scratch/ECHOAnalysis.jl/Manifest.toml`
  [b4f34e82] - Distances v0.9.0
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
  [3cdcf5f2] ↓ RecipesBase v1.0.2 ⇒ v0.8.0
  [348f2d5d] ↓ SpatialEcology v0.9.5 ⇒ v0.7.0
  [2913bbd2] ↓ StatsBase v0.33.0 ⇒ v0.32.2

This is just a personal package and the issue is easily solved by adding Microbiome 0.6.4 to the deps, so it's not urgent for me, but seems like an odd bug. I'm wondering if it might be a downstream consequence of the move to the BioJulia registry and then back to General, coupled with something I did here: JuliaRegistries/General#10324

Steps to reproduce:

$ git clone git@github.com:Klepac-Ceraj-Lab/ECHOAnalysis.jl.git # current commit 4f3e59b
Cloning into 'ECHOAnalysis.jl'...
remote: Enumerating objects: 233, done.
remote: Counting objects: 100% (233/233), done.
remote: Compressing objects: 100% (118/118), done.
remote: Total 1840 (delta 118), reused 193 (delta 83), pack-reused 1607
Receiving objects: 100% (1840/1840), 12.91 MiB | 7.24 MiB/s, done.
Resolving deltas: 100% (1134/1134), done.
$ cd ECHOAnalysis.jl 
$ julia --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0 (2020-08-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(ECHOAnalysis) pkg> add Microbiome@0.6.4
  Resolving package versions...
Updating `~/repos/scratch/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
Updating `~/repos/scratch/ECHOAnalysis.jl/Manifest.toml`
  [b4f34e82] + Distances v0.9.0
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
  [3cdcf5f2] ↑ RecipesBase v0.8.0 ⇒ v1.0.2
  [348f2d5d] ↑ SpatialEcology v0.7.0 ⇒ v0.9.5
  [2913bbd2] ↑ StatsBase v0.32.2 ⇒ v0.33.0

(ECHOAnalysis) pkg> up
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
Updating `~/repos/scratch/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
Updating `~/repos/scratch/ECHOAnalysis.jl/Manifest.toml`
  [b4f34e82] - Distances v0.9.0
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
  [3cdcf5f2] ↓ RecipesBase v1.0.2 ⇒ v0.8.0
  [348f2d5d] ↓ SpatialEcology v0.9.5 ⇒ v0.7.0
  [2913bbd2] ↓ StatsBase v0.33.0 ⇒ v0.32.2

[KristofferC]

If I remove ProgressMeter from the project, it stops happening:

(ECHOAnalysis) pkg> rm ProgressMeter
Updating `~/Downgrade/ECHOAnalysis.jl/Project.toml`
  [92933f4c] - ProgressMeter v1.3.2
Updating `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`
  [92933f4c] - ProgressMeter v1.3.2

(ECHOAnalysis) pkg> add Microbiome@0.6.4
  Resolving package versions...
Updating `~/Downgrade/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
Updating `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
  [3cdcf5f2] ↑ RecipesBase v0.8.0 ⇒ v1.0.2
  [348f2d5d] ↑ SpatialEcology v0.7.0 ⇒ v0.9.5
  [2913bbd2] ↑ StatsBase v0.32.2 ⇒ v0.33.0

(ECHOAnalysis) pkg> up
No Changes to `~/Downgrade/ECHOAnalysis.jl/Project.toml`
No Changes to `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`

(ECHOAnalysis) pkg> add ProgressMeter
  Resolving package versions...
Updating `~/Downgrade/ECHOAnalysis.jl/Project.toml`
  [92933f4c] + ProgressMeter v1.3.2
Updating `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`
  [92933f4c] + ProgressMeter v1.3.2

(ECHOAnalysis) pkg> up
Updating `~/Downgrade/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
Updating `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
  [3cdcf5f2] ↓ RecipesBase v1.0.2 ⇒ v0.8.0
  [348f2d5d] ↓ SpatialEcology v0.9.5 ⇒ v0.7.0
  [2913bbd2] ↓ StatsBase v0.33.0 ⇒ v0.32.2

(ECHOAnalysis) pkg> rm ProgressMeter
Updating `~/Downgrade/ECHOAnalysis.jl/Project.toml`
  [92933f4c] - ProgressMeter v1.3.2
Updating `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`
  [92933f4c] - ProgressMeter v1.3.2

(ECHOAnalysis) pkg> up
Updating `~/Downgrade/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
Updating `~/Downgrade/ECHOAnalysis.jl/Manifest.toml`
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
  [3cdcf5f2] ↑ RecipesBase v0.8.0 ⇒ v1.0.2
  [348f2d5d] ↑ SpatialEcology v0.7.0 ⇒ v0.9.5
  [2913bbd2] ↑ StatsBase v0.32.2 ⇒ v0.33.0

Any idea about this @carlobaldassi?

[KristofferC]

This also seems to happen back to 1.3 so it isn't something new we introduced.

[KristofferC]

This looks like the resolver accuracy is kicking in:

(ECHOAnalysis) pkg> up
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
Updating `~/JuliaTests/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
Updating `~/JuliaTests/ECHOAnalysis.jl/Manifest.toml`
  [b4f34e82] - Distances v0.9.0
  [3bd8f0ae] ↓ Microbiome v0.6.4 ⇒ v0.6.1
  [3cdcf5f2] ↓ RecipesBase v1.0.2 ⇒ v0.8.0
  [348f2d5d] ↓ SpatialEcology v0.9.5 ⇒ v0.7.0
  [2913bbd2] ↓ StatsBase v0.33.0 ⇒ v0.32.2

julia> ENV["JULIA_PKGRESOLVE_ACCURACY"] = 2
2

(ECHOAnalysis) pkg> up
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
Updating `~/JuliaTests/ECHOAnalysis.jl/Project.toml`
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
Updating `~/JuliaTests/ECHOAnalysis.jl/Manifest.toml`
  [b4f34e82] + Distances v0.9.0
  [3bd8f0ae] ↑ Microbiome v0.6.1 ⇒ v0.6.4
  [3cdcf5f2] ↑ RecipesBase v0.8.0 ⇒ v1.0.2
  [348f2d5d] ↑ SpatialEcology v0.7.0 ⇒ v0.9.5
  [2913bbd2] ↑ StatsBase v0.32.2 ⇒ v0.33.0

Perhaps we should bump the default accuracy. I'll do some benchmarks and see the effect.

[kescobo]

Interesting - thanks! This package has a ton of dependencies (some of which are likely unnecessary at this stage). Using just Microbiome and ProgressMeter, it seems to install the most recent version:

(scratch) pkg> add Microbiome ProgressMeter
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Updating `~/repos/scratch/Project.toml`
  [3bd8f0ae] + Microbiome v0.6.4
  [92933f4c] + ProgressMeter v1.3.2

I mention this to say that I would not necessarily use this package as a reason to change a default that slows performance for everyone, based on this example.

[andreasnoack]

I'm seeing similar behavior even when setting the resolver accuracy to seven

(TestJuliaRun) pkg> add StatsBase
  Resolving package versions...
Updating `~/TestJuliaRun/Project.toml`
  [2913bbd2] + StatsBase v0.33.1
Updating `~/TestJuliaRun/Manifest.toml`
  [2913bbd2] ↑ StatsBase v0.32.0 ⇒ v0.33.1
(TestJuliaRun) pkg> rm StatsBase
Updating `~/TestJuliaRun/Project.toml`
  [2913bbd2] - StatsBase v0.33.1
No Changes to `~/TestJuliaRun/Manifest.toml`
(TestJuliaRun) pkg> up
   Updating registry at `~/.julia/registries/General`
   Updating registry at `~/.julia/registries/JuliaComputingRegistry`
   Updating registry at `~/.julia/registries/PumasRegistry`
No Changes to `~/TestJuliaRun/Project.toml`
Updating `~/TestJuliaRun/Manifest.toml`
  [2913bbd2] ↓ StatsBase v0.33.1 ⇒ v0.32.0

[KristofferC]

Here is another example:

(jl_5JO6A6) pkg> add DifferentialEquations
...

(jl_5JO6A6) pkg> up
    Updating registry at `~/.julia/registries/General`
  No Changes to `/tmp/jl_5JO6A6/Project.toml`
  No Changes to `/tmp/jl_5JO6A6/Manifest.toml`

(jl_5JO6A6) pkg> add ModelingToolkit@6
   Resolving package versions...
    Updating `/tmp/jl_5JO6A6/Project.toml`
  [961ee093] + ModelingToolkit v6.0.0
    Updating `/tmp/jl_5JO6A6/Manifest.toml`
  [fb6a15b2] + CloseOpenIntervals v0.1.1
  [754358af] + DEDataArrays v0.2.0
  [2b5f629d] ↑ DiffEqBase v6.70.0 ⇒ v6.71.1
  [31c24e10] ↑ Distributions v0.24.18 ⇒ v0.25.11
  [1a297f60] ↑ FillArrays v0.11.9 ⇒ v0.12.1
  [961ee093] ↑ ModelingToolkit v5.26.0 ⇒ v6.0.0
  [d236fae5] + PreallocationTools v0.1.0
  [f2c3362d] ↑ RecursiveFactorization v0.1.13 ⇒ v0.2.1
  [0c5d862f] ↑ Symbolics v1.4.2 ⇒ v2.1.0
  [d5829a12] + TriangularSolve v0.1.2

@carlobaldassi, do you have any idea why this would happen? Why would ModelingToolkit not get installed at 6.0 when it can be done without downgrading anything? Also:

julia> ENV["JULIA_PKGRESOLVE_ACCURACY"] = 2
2

(jl_UvKmhf) pkg> up
    Updating registry at `~/.julia/registries/General`
  No Changes to `/tmp/jl_UvKmhf/Project.toml`
  No Changes to `/tmp/jl_UvKmhf/Manifest.toml`

julia> ENV["JULIA_PKGRESOLVE_ACCURACY"] = 3
3

(jl_UvKmhf) pkg> up
    Updating registry at `~/.julia/registries/General`
   Installed BandedMatrices ─ v0.15.25
  No Changes to `/tmp/jl_UvKmhf/Project.toml`
    Updating `/tmp/jl_UvKmhf/Manifest.toml`
  [4c555306] ↓ ArrayLayouts v0.7.3 ⇒ v0.4.12
  [aae01518] ↓ BandedMatrices v0.16.10 ⇒ v0.15.25
  [1a297f60] ↓ FillArrays v0.11.9 ⇒ v0.10.2

So increasing the accuracy, caused packages to only downgrade. Hmm.

Removing some compat entries seems to "fix" it:

shell> pwd
/home/kc/JuliaPkgs/Pkg.jl/dev/ParameterizedFunctions

shell> git --no-pager diff
diff --git a/Project.toml b/Project.toml
index a92c097..b537062 100644
--- a/Project.toml
+++ b/Project.toml
@@ -18,7 +18,7 @@ DataStructures = "0.17, 0.18"
 DiffEqBase = "6.5"
 DocStringExtensions = "0.8"
 Latexify = "0.14, 0.15"
-ModelingToolkit = "4.0, 5, 6"
+ModelingToolkit = "6"
 Reexport = "0.2, 1.0"
 SciMLBase = "1.3"
 julia = "1.3"

(Pkg) pkg> dev .
   Resolving package versions...
    Updating `~/JuliaPkgs/Pkg.jl/Project.toml`
  [65888b18] ~ ParameterizedFunctions v5.11.0 ⇒ v5.11.0 `dev/ParameterizedFunctions`
    Updating `~/JuliaPkgs/Pkg.jl/Manifest.toml`
  [fb6a15b2] + CloseOpenIntervals v0.1.1
  [754358af] + DEDataArrays v0.2.0
  [2b5f629d] ↑ DiffEqBase v6.70.0 ⇒ v6.71.1
  [31c24e10] ↑ Distributions v0.24.18 ⇒ v0.25.11
  [1a297f60] ↑ FillArrays v0.11.9 ⇒ v0.12.1
  [961ee093] ↑ ModelingToolkit v5.26.0 ⇒ v6.0.0
  [65888b18] ~ ParameterizedFunctions v5.11.0 ⇒ v5.11.0 `dev/ParameterizedFunctions`
  [d236fae5] + PreallocationTools v0.1.0
  [f2c3362d] ↑ RecursiveFactorization v0.1.13 ⇒ v0.2.1
  [0c5d862f] ↑ Symbolics v1.4.2 ⇒ v2.1.0
  [d5829a12] + TriangularSolve v0.1.2

However, even with this, we are still not optimal:

(jl_k4iWr4) pkg> add ArrayInterface@3.1.23
   Resolving package versions...
    Updating `/tmp/jl_k4iWr4/Project.toml`
 [4fba245c] + ArrayInterface v3.1.23
    Updating `/tmp/jl_k4iWr4/Manifest.toml`
 [4fba245c] ↑ ArrayInterface v3.1.17 ⇒ v3.1.23
 [aedffcd0] ↑ Static v0.2.5 ⇒ v0.3.0

[carlobaldassi]

I'll have a look as soon as I can (it will likely take a few days at least though).

[KristofferC]

@carlobaldassi, did you find anything?

[carlobaldassi]

Not much yet, sorry. I had been able to reproduce the issue with ModelingToolkit a week ago or so, however right now it's no longer present. The issue with ArrayInterface is still reproducible though, so I may use that.
My guess (still need to verify this) of what's going on from inspecting the dependencies is that the solver finds a sub-optimal solution (which is always a possibility) and then the algorithm that tries to do a local optimization (enforce_optimality! function) fails because it's too naive.
If so, I think that the best course of action would be improving the local optimizer, that shouldn't be too hard.
Another good idea in my opinion (I had this since a long time) would be to feed the resolver with a reduced graph: after the graph has been generated, pruned etc, we could attempt a solution with just a few of the highest available versions of each package. That should help the solver a lot, probably work fine nearly all the time, and be quicker on average. The full graph could be then used as a fallback in case of failure.
Anyway, hopefully I'll explore these possibilities in the next few days, especially the first one. I just need to find a few hours somewhere...