Julia v1.8.0-beta1 shows outdated packages but does not update them

[ranocha]

Here's what I get:

julia> versioninfo()
Julia Version 1.8.0-beta1
Commit 7b711ce699 (2022-02-23 15:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 12 × Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 12 virtual cores
Environment:
  LD_LIBRARY_PATH = :/usr/local/lib
  JULIA_NUM_THREADS = 1
  JULIA_EDITOR = code

julia> using Pkg; Pkg.activate(temp=true)
  Activating new project at `/tmp/jl_mf0QGH`

julia> Pkg.add(["Trixi", "LoopVectorization"])
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `/tmp/jl_mf0QGH/Project.toml`
⌃ [bdcacae8] + LoopVectorization v0.12.101
  [a7f1ee26] + Trixi v0.4.23
    Updating `/tmp/jl_mf0QGH/Manifest.toml`
  [621f4979] + AbstractFFTs v1.1.0
  [79e6a3ab] + Adapt v3.3.3
  [dce04be8] + ArgCheck v2.3.0
⌅ [4fba245c] + ArrayInterface v3.2.2
  [4c555306] + ArrayLayouts v0.7.10
  [b99e7846] + BinaryProvider v0.5.10
  [62783981] + BitTwiddlingConvenienceFunctions v0.1.3
⌅ [d43a6710] + CBinding v0.9.4
  [2a0fbf3d] + CPUSummary v0.1.8
  [d360d2e6] + ChainRulesCore v1.13.0
  [9e997f8a] + ChangesOfVariables v0.1.2
  [fb6a15b2] + CloseOpenIntervals v0.1.5
  [da1fd8a2] + CodeTracking v1.0.7
  [35d6a980] + ColorSchemes v3.17.1
  [3da002f7] + ColorTypes v0.11.0
  [5ae59095] + Colors v0.12.8
  [38540f10] + CommonSolve v0.2.0
  [bbf7d656] + CommonSubexpressions v0.3.0
  [34da2185] + Compat v3.41.0
  [187b0558] + ConstructionBase v1.3.0
  [587fd27a] + CovarianceEstimation v0.2.7
  [717857b8] + DSP v0.7.5
  [9a962f9c] + DataAPI v1.9.0
  [864edb3b] + DataStructures v0.18.11
  [e2d170a0] + DataValueInterfaces v1.0.0
  [163ba53b] + DiffResults v1.0.3
  [b552c78f] + DiffRules v1.10.0
  [ffbed154] + DocStringExtensions v0.8.6
  [da5c29d0] + EllipsisNotation v1.3.0
  [e2ba6199] + ExprTools v0.1.8
  [7a1cc6ca] + FFTW v1.4.6
  [442a2c76] + FastGaussQuadrature v0.4.9
  [057dd010] + FastTransforms v0.13.6
  [1a297f60] + FillArrays v0.13.0
  [53c48c17] + FixedPointNumbers v0.8.4
  [f6369f11] + ForwardDiff v0.10.25
  [5c1252a2] + GeometryBasics v0.4.1
  [f67ccb44] + HDF5 v0.16.2
  [3e5b6fbb] + HostCPUFeatures v0.1.6
  [0e44f5e4] + Hwloc v2.0.0
  [615f187c] + IfElse v0.1.1
  [3587e190] + InverseFunctions v0.1.2
  [92d709cd] + IrrationalConstants v0.1.1
  [c8e1da08] + IterTools v1.4.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.4.1
  [2c470bb0] + Kronecker v0.5.1
  [10f19ff3] + LayoutPointers v0.1.5
  [7a12625a] + LinearMaps v3.5.1
  [2ab3a3ac] + LogExpFunctions v0.3.6
⌃ [bdcacae8] + LoopVectorization v0.12.101
  [da04e1cc] + MPI v0.19.2
  [1914dd2f] + MacroTools v0.5.9
  [d125e4d3] + ManualMemory v0.1.8
  [e1d29d7a] + Missings v1.0.2
  [46d2c3a1] + MuladdMacro v0.2.2
  [d8a4904e] + MutableArithmetics v1.0.0
⌅ [77ba4419] + NaNMath v0.3.7
  [356022a1] + NamedDims v0.2.46
  [7aca2e03] + NodesAndModes v0.7.1
  [6fd5a793] + Octavian v0.3.12
  [6fe1bfb0] + OffsetArrays v1.10.8
  [bac558e1] + OrderedCollections v1.4.1
  [7d669430] + P4est v0.3.0
  [995b91a9] + PlotUtils v1.1.3
⌃ [f517fe37] + Polyester v0.6.4
  [1d0040c9] + PolyesterWeave v0.1.4
  [c74db56a] + PolynomialBases v0.4.11
  [f27b6e38] + Polynomials v3.0.0
  [21216c6a] + Preferences v1.2.4
  [3cdcf5f2] + RecipesBase v1.2.1
  [731186ca] + RecursiveArrayTools v2.24.2
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [3cdde19b] + SIMDDualNumbers v0.1.0
  [94e857df] + SIMDTypes v0.1.0
  [476501e8] + SLEEFPirates v0.6.30
  [0bca4576] + SciMLBase v1.26.3
  [efcf1570] + Setfield v0.8.2
  [a2af1166] + SortingAlgorithms v1.0.1
  [276daf66] + SpecialFunctions v2.1.4
  [472ebc20] + StartUpDG v0.12.6
⌅ [aedffcd0] + Static v0.4.1
  [90137ffa] + StaticArrays v1.4.1
  [82ae8749] + StatsAPI v1.2.1
  [2913bbd2] + StatsBase v0.33.16
  [d1fa6d79] + StrideArrays v0.1.15
  [7792a7ef] + StrideArraysCore v0.2.12
  [09ab397b] + StructArrays v0.6.5
  [9f78cca6] + SummationByPartsOperators v0.5.13
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.6.1
⌅ [8290d209] + ThreadingUtilities v0.4.7
  [a759f4b9] + TimerOutputs v0.5.15
  [c751599d] + ToeplitzMatrices v0.7.0
  [a2a6695c] + TreeViews v0.3.0
  [f7e6ffb2] + Triangulate v2.1.2
  [981d1d27] + TriplotBase v0.1.0
  [808ab39a] + TriplotRecipes v0.1.0
  [a7f1ee26] + Trixi v0.4.23
  [3a884ed6] + UnPack v1.0.2
  [9602ed7d] + Unrolled v0.1.3
  [3d5dd08c] + VectorizationBase v0.21.24
  [33b4df10] + VectorizedRNG v0.2.14
  [700de1a5] + ZygoteRules v0.2.2
  [5ae413db] + EarCut_jll v2.2.3+0
  [f5851436] + FFTW_jll v3.3.10+0
  [34b6f7d7] + FastTransforms_jll v0.5.4+0
  [0234f1f7] + HDF5_jll v1.12.1+0
  [e33a78d0] + Hwloc_jll v2.7.0+0
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+2
  [856f044c] + MKL_jll v2022.0.0+0
  [7cb0a576] + MPICH_jll v4.0.1+0
  [9237b28f] + MicrosoftMPI_jll v10.1.3+2
  [fe0851c0] + OpenMPI_jll v4.1.2+0
  [458c3c95] + OpenSSL_jll v1.1.13+0
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [6b5a15aa] + P4est_jll v2.8.0+0
  [5639c1d2] + Triangle_jll v1.6.1+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.0+0
  [781609d7] + GMP_jll v6.2.1+1
  [deac9b47] + LibCURL_jll v7.81.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [3a97d323] + MPFR_jll v4.1.1+1
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.17+2
  [05823500] + OpenLibm_jll v0.8.1+0
  [83775a58] + Zlib_jll v1.2.12+1
  [8e850b90] + libblastrampoline_jll v5.0.1+0
  [8e850ede] + nghttp2_jll v1.41.0+1
  [3f19e933] + p7zip_jll v16.2.1+1
        Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ cannot be upgraded. To see why use `status --outdated`

julia> Pkg.status()
Status `/tmp/jl_mf0QGH/Project.toml`
⌃ [bdcacae8] LoopVectorization v0.12.101
  [a7f1ee26] Trixi v0.4.23
Info Packages marked with ⌃ have new versions available

julia> Pkg.update()
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `/tmp/jl_mf0QGH/Project.toml`
  No Changes to `/tmp/jl_mf0QGH/Manifest.toml`

julia> Pkg.status()
Status `/tmp/jl_mf0QGH/Project.toml`
⌃ [bdcacae8] LoopVectorization v0.12.101
  [a7f1ee26] Trixi v0.4.23
Info Packages marked with ⌃ have new versions available

julia> Pkg.update("LoopVectorization")
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `/tmp/jl_mf0QGH/Project.toml`
  No Changes to `/tmp/jl_mf0QGH/Manifest.toml`

julia> Pkg.status()
Status `/tmp/jl_mf0QGH/Project.toml`
⌃ [bdcacae8] LoopVectorization v0.12.101
  [a7f1ee26] Trixi v0.4.23
Info Packages marked with ⌃ have new versions available

The output of Pkg.status() shows that there is a newer version of LoopVectorization.jl but it does not update it.

If I activate a new environment and install only LoopVectorization.jl, everything is fine:

julia> Pkg.activate(temp=true)
  Activating new project at `/tmp/jl_xsHKgI`

julia> Pkg.add("LoopVectorization")
   Resolving package versions...
    Updating `/tmp/jl_xsHKgI/Project.toml`
  [bdcacae8] + LoopVectorization v0.12.102
    Updating `/tmp/jl_xsHKgI/Manifest.toml`
  [79e6a3ab] + Adapt v3.3.3
⌅ [4fba245c] + ArrayInterface v3.2.2
  [62783981] + BitTwiddlingConvenienceFunctions v0.1.3
  [2a0fbf3d] + CPUSummary v0.1.8
  [d360d2e6] + ChainRulesCore v1.13.0
  [9e997f8a] + ChangesOfVariables v0.1.2
  [fb6a15b2] + CloseOpenIntervals v0.1.5
  [bbf7d656] + CommonSubexpressions v0.3.0
  [34da2185] + Compat v3.41.0
  [163ba53b] + DiffResults v1.0.3
  [b552c78f] + DiffRules v1.10.0
  [ffbed154] + DocStringExtensions v0.8.6
  [f6369f11] + ForwardDiff v0.10.25
  [3e5b6fbb] + HostCPUFeatures v0.1.6
  [0e44f5e4] + Hwloc v2.0.0
  [615f187c] + IfElse v0.1.1
  [3587e190] + InverseFunctions v0.1.2
  [92d709cd] + IrrationalConstants v0.1.1
  [692b3bcd] + JLLWrappers v1.4.1
  [10f19ff3] + LayoutPointers v0.1.5
  [2ab3a3ac] + LogExpFunctions v0.3.6
  [bdcacae8] + LoopVectorization v0.12.102
  [1914dd2f] + MacroTools v0.5.9
  [d125e4d3] + ManualMemory v0.1.8
⌅ [77ba4419] + NaNMath v0.3.7
  [6fe1bfb0] + OffsetArrays v1.10.8
  [1d0040c9] + PolyesterWeave v0.1.4
  [21216c6a] + Preferences v1.2.4
  [ae029012] + Requires v1.3.0
  [3cdde19b] + SIMDDualNumbers v0.1.0
  [94e857df] + SIMDTypes v0.1.0
  [476501e8] + SLEEFPirates v0.6.30
  [276daf66] + SpecialFunctions v2.1.4
⌅ [aedffcd0] + Static v0.4.1
  [90137ffa] + StaticArrays v1.4.1
  [8290d209] + ThreadingUtilities v0.5.0
  [3a884ed6] + UnPack v1.0.2
  [3d5dd08c] + VectorizationBase v0.21.24
  [e33a78d0] + Hwloc_jll v2.7.0+0
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.0+0
  [deac9b47] + LibCURL_jll v7.81.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.17+2
  [05823500] + OpenLibm_jll v0.8.1+0
  [83775a58] + Zlib_jll v1.2.12+1
  [8e850b90] + libblastrampoline_jll v5.0.1+0
  [8e850ede] + nghttp2_jll v1.41.0+1
  [3f19e933] + p7zip_jll v16.2.1+1
        Info Packages marked with ⌅ have new versions available but cannot be upgraded. To see why use `status --outdated`

julia> Pkg.status()
Status `/tmp/jl_xsHKgI/Project.toml`
  [bdcacae8] LoopVectorization v0.12.102

[KristofferC]

Just because there is nothing holding back a package doesn't mean that up will update it. That is because updating the package might cause other packages to downgrade. My guess is that if you write pkg> add LoopVectorization@0.12.102 in the original environment, it will happily install, but some other package will downgrade because of it.

In those cases, if you specifically want a version of some package you either need to add it by version (like shown above) or use the [compat] entry in the project file.

[ranocha]

Right, pkg> add LoopVectorization@0.12.102 downgrades other packages. Maybe it would be good to add a note to the info line

        Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ cannot be upgraded. To see why use `status --outdated`

For me, it was at least a bit confusing to see that Pkg tells me there are new versions but refuses to update them easily. Maybe something along the lines of

        Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ cannot be upgraded. To see why use `status --outdated`. Note that not all packages with new versions (⌃) will be updated if this would downgrade other dependencies.

[IanButterworth]

I wonder if what's needed is something like

pkg> up
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `/tmp/jl_mf0QGH/Project.toml`
  No Changes to `/tmp/jl_mf0QGH/Manifest.toml`

pkg> up LoopVectorization
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `/tmp/jl_mf0QGH/Project.toml`
  No Changes to `/tmp/jl_mf0QGH/Manifest.toml`
 Info There is a newer version of LoopVectorization available, but at a cost of downgrading an unbalanced number of packages.
      Use `pkg> add LoopVectorization@0.12.102` to force installation.

i.e. the extra info is only given when an explicit package up is requested

[ranocha]

That would be even better 👍

[DilumAluthge]

I wonder if what's needed is something like

pkg> up
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `/tmp/jl_mf0QGH/Project.toml`
  No Changes to `/tmp/jl_mf0QGH/Manifest.toml`

pkg> up LoopVectorization
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `/tmp/jl_mf0QGH/Project.toml`
  No Changes to `/tmp/jl_mf0QGH/Manifest.toml`
 Info There is a newer version of LoopVectorization available, but at a cost of downgrading an unbalanced number of packages.
      Use `pkg> add LoopVectorization@0.12.102` to force installation.

i.e. the extra info is only given when an explicit package up is requested

I disagree with this. If this change is made, then:

1. User does ] up.
2. User assumes that all of their direct dependencies are at the latest version.
3. User is surprised when this assumption is false.

We routinely see users on Discourse that assume that doing ] up will update all of their direct dependencies are at the latest version.

I think that if ] up results in one or more direct dependencies that are not at the latest version, we should always print a warning, regardless of the specific reason why the dependency is not at the latest version.

[ranocha]

I have been using Julia for many years. I know that I will not necessarily get the newest versions of all direct dependencies - since upgrading some of them may downgrade other packages. However, the new sign indicating that a direct dependency can be upgraded confused me quite a bit. So I agree with @DilumAluthge - it will probably avoid confusion if such warnings are shown. Another related option might be to introduce a third symbol indicating packages that have new versions that could be installed (so not ⌅ with its current meaning) but will not be installed since that would downgrade other packages.

[timholy]

Can you make a specific suggestion for what that symbol would be? (It's often quite hard to come up with candidates.)

And what would ⌅ now do? You can always install the latest version of a package in isolation, it's only incompatibilities with other packages that prevent it. So I'm not sure what this new category really means.

[ranocha]

That's the problem - I also do not really have a good suggestion, sorry.

1. One option may be to also use ⌅ for packages with newer versions that will not be installed by ] up because other packages would be downgraded. That would be totally fine in my opinion. In this case, it would be good to add a warning as described by @DilumAluthge above
2. Another option may be to use
   • ⌅ for packages with newer versions that will not be installed by ] up because other packages would be downgraded, i.e., a soft reason for keeping an old version ("I do not want to do this")
   • ⌆ for packages with newer versions that will not be installed by ] up because of version conflicts etc. (basically the current meaning of ⌅?), i.e., a hard reason for keeping an old version ("I really cannot do this")

[timholy]

How about my second point? I don't understand what the new symbol means that's actually different from ⌅? What is a "version conflict"? Do you mean just with Julia itself? That's the only "hard" conflict there is; all other version numbers are negotiable.

[KristofferC]

I think a good compromise is that after up and there are packages that are still not at the latest version to show something (but maybe a bit terser) as in #3012 (comment).

[ranocha]

How about my second point? I don't understand what the new symbol means that's actually different from ⌅? What is a "version conflict"? Do you mean just with Julia itself? That's the only "hard" conflict there is; all other version numbers are negotiable.

My first guess after reading the (not really concrete) statement

those with ⌅ cannot be upgraded

was that there are probably some version conflicts preventing an upgrade because of explicit compat bounds of the project or other dependencies, e.g.,

• Project.toml restricts package A to version v0.3 but v0.4 is available
• Project.toml requires packages A and B; package A restricts package B to v0.3 but v0.4 is available

In the end, my point is that

Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ cannot be upgraded.

sounds to me like those package with ⌃ can be upgraded (with ] up). Any additional hint that this is not the case would be helpful, I think. (At least it would have prevented me from opening this issue and wasting your time - sorry for that and thanks to everybody implementing great stuff in Julia 🙂).