Add Statistics package and compat for it to packages

[nalimilan]

Part of JuliaLang/julia#46501. Mirrors #66039.

[nalimilan]

Anything else to do?

[KristofferC]

I would suggest testing out the "final state" (this PR, the juliaPR and the Pkg PR together) and seeing that things behave as expected. Other than that, I think this should be good.

[nalimilan]

I've just tried that, it seems to work: I can use Statistics without installing it, install it (and then the new version is loaded), and install packages that depend on it such as StatsBase. Anything else to test?

[KristofferC]

That sounds good. Maybe also just test this PR in isolation (like on julia 1.9) to make sure it doesn't cause any weirdness. It shouldn't but hey.

[nalimilan]

Looks good too. I'm just a bit surprised to this this on Julia 1.9:

(jl_C9Co6h) pkg> add Statistics
   Resolving package versions...
    Updating `/tmp/jl_C9Co6h/Project.toml`
  [10745b16] + Statistics v1.9.0
    Updating `/tmp/jl_C9Co6h/Manifest.toml`
[...]

On 1.8 I see [10745b16] + Statistics without the v1.9.0. But that probably doesn't matter.

[nalimilan]

Good to go?

[vchuravy]

@nalimilan Can you rebase your changes again?
Otherwise LGTM.

[nalimilan]

Done.

[KristofferC]

On 1.8 I see [10745b16] + Statistics without the v1.9.0. But that probably doesn't matter.

Hmm, it seems like the behavior for DelimitedFiles is the same. Probably doesn't matter in practice. And 1.9 will be obsolete in not too long time anyway.

[nalimilan]

Re-done. :-)

[vchuravy]

Slightly nerve racking PR

[baumgold]

@nalimilan - I believe there are some issues with this PR. Specifically it prevents the StaticArrays.jl unit-tests from running when using Julia v1.1 (StaticArrays v1.3.6). See error below.

Specifically, I think the Statistics compat for StaticArrays v1-1.3 should be
1 - 1.11 rather than 1 - 1.10 since Julia v1.1 seems to ship with Statistics v1.11.

Likely this problem is not limited to StaticArrays but I don't know the exact extent of the issue. Maybe we should rollback this PR until the issue is fully understood and resolved?

$ julia +1.1
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.1 (2019-05-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.1) pkg> st
    Status `~/.julia/environments/v1.1/Project.toml`
  [90137ffa] StaticArrays v1.3.6

(v1.1) pkg> test StaticArrays
   Testing StaticArrays
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package StaticArrays [90137ffa]:
 StaticArrays [90137ffa] log:
 ├─possible versions are: [0.8.0-0.8.3, 0.9.0-0.9.2, 0.10.0, 0.10.2-0.10.3, 0.11.0-0.11.1, 0.12.0-0.12.6, 1.0.0-1.0.1, 1.1.0-1.1.3, 1.2.0-1.2.13, 1.3.0-1.3.6, 1.4.0-1.4.7, 1.5.0-1.5.26, 1.6.0-1.6.2] or uninstalled
 ├─restricted to versions 1.3.6 by an explicit requirement, leaving only versions 1.3.6
 └─restricted by compatibility requirements with Statistics [10745b16] to versions: [0.12.0-0.12.6, 1.5.0-1.5.26, 1.6.0-1.6.2] or uninstalled — no versions left
   └─Statistics [10745b16] log:
     ├─possible versions are: 1.11.0 or uninstalled
     └─Statistics [10745b16] is fixed to version 1.11.0+

[nalimilan]

Hmmm. Julia 1.1 ships with a Statistics stdlib module that has no version and Pkg is trying to install version 1.11 which is the only version in the registry. This is a bad interaction with #86950, which disallowed Julia 1.10 for older StaticArrays releases. I'm not sure why the error happens only when testing and not when adding the package.

@KristofferC It seems that dropping the compat bounds for Statistics fixes the problem. Is there any advantage in having these bounds in addition to those on Julia versions, which should be equivalent for older Julia releases (which are not supported by Statistics 1.11 anyway)? Otherwise, maybe we should register on older Julia releases a version of Statistics that does nothing or merely reexports functions from the stdlib module?

[KristofferC]

Hmmm. Julia 1.1 ships with a Statistics stdlib module that has no version and Pkg is trying to install version 1.11 which is the only version in the registry.

Hmm, I don't see why a Julia with Statistics as an (non-upgradable) stdlib would look at the registry information for Statistics at all. That seems like an issue with Pkg...

Registering an older version (a 1.9 version with perhaps the same content as v1.9 Statistics has) could possibly fix it indeed, if the issue is with not having a pre 1.10 version available.

[nalimilan]

I just tried registering Statistics 1.8 as it works on all Julia versions since 1.0, but that doesn't fix the problem. It seems that the mere existence of 1.11 is enough to confuse the resolver on Julia 1.0 to 1.3. If I remove that version and keep only 1.8, the problem is gone.

I've checked manually that the only two packages that have an upper bound on Statistics are StaticArrays and FillArrays. So I've filed #90310 to adjust their bounds, that sounds like the simplest solution.

[stemann]

This also broke testing on Julia 1.3 for other packages, e.g. https://github.com/IHPSystems/ImageAnnotations.jl : Statistics [10745b16] log: possible versions are: 1.11.0 or uninstalled