std not re-exported?

[robsmith11]

I'm confused why some functions are re-exported, but others are not:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.3.0-DEV.515 (2019-07-05)
 _/ |\__'_|_|_|\__'_|  |  Commit 2350c04091* (4 days old master)
|__/                   |

julia> using StatsBase

julia> StatsBase.mean(rand(10))
0.43976198608132133

julia> StatsBase.std(rand(10))
0.27238371213261153

julia> mean(rand(10))
0.4492048091159031

julia> std(rand(10))
ERROR: UndefVarError: std not defined
Stacktrace:
 [1] top-level scope at REPL[5]:1

Am I missing something obvious? I see them both listed in StatsBase.jl:

$ grep std ~/.julia/packages/StatsBase/rTkaz/src/StatsBase.jl
import Statistics: mean, mean!, var, varm, varm!, std, stdm, cov, covm,

[nalimilan]

That's not really intended, it's just because we use @deprecate for a method for mean. But I guess it would make sense to reexport all function to which we add methods.

[robsmith11]

Ah I see.. I was thinking that I could use StatsBase.jl as a superset of the stdlib Statistics module, which is quite limited. I guess I can just import them both. :)

[EricForgy]

FYI, I bumped into this too and was (am) super confused.

Why not re-export Statistics.jl? It is probably safe to assume that using StatsBase should give a std. I had to sift through the source code to figure out what is going on. The StatsBase.jl docs refer to Statistics.jl docs, which is also confusing 🤔

[Edit: Since the docs refer to Statistics.std, etc, I think StatsBase.jl should at least export Statistics if not re-export. I can submit a PR either way if it would be welcome.]

[nalimilan]

Yes I think we should reexport Statistics. Feel free to make a PR. One question is whether/how we should document functions that live in Statistics to make them easier to discover. Maybe at least a list with a link to the Julia manual would be useful.

[EricForgy]

One question is whether/how we should document functions that live in Statistics to make them easier to discover.

I'm not sure what you mean. One of the things that confused me was the StatsBase docs already include docs for Statistics.jl. For example:

https://juliastats.org/StatsBase.jl/stable/scalarstats/

[nalimilan]

No, these only document the methods that take a weights vector.

[EricForgy]

I see. I'm new to using stats in Julia and I get the impression the general situation is less than ideal for this basic stuff.

The standard library Statistics.jl seems sorely incomplete and it is an unfortunate name grab. What is called StatsBase.jl should probably just be Statistics.jl and the standard lib should be called StatsCore.jl or StatsLib.jl or something. What is the current thinking about all this? Is there a discussion somewhere? I can also see an argument to put all of StatsBase.jl into the standard library Statistics.jl, but that should probably wait until it is a separate repo (if it isn't).

I'm willing to put some work into this if there is a vision. What is the vision? Is the status quo ok for everyone?

It hurts my brain thinking I need to import multiple StatXXX.jl packages to do simple things, but maybe it is just me.

[nalimilan]

Statistics.jl has had its separate repo for some time, but so far nobody has found the time to finish porting features from StatsBase. See https://github.com/JuliaLang/Statistics.jl/pull/2 and https://github.com/JuliaLang/Statistics.jl/pull/31.

[EricForgy]

I'm looking at this. It seems master branch of Statistics.jl is failing tests. What branch is actually in v1.5.3? What branch is going into v1.6?

If I wanted to help, should I start with https://github.com/JuliaLang/Statistics.jl/pull/2 or https://github.com/JuliaLang/Statistics.jl/pull/31 or start fresh from master?

Is the intention to move all of StatsBase.jl into Statistics.jl?

[nalimilan]

CI should be fixed now (it needed https://github.com/JuliaLang/Statistics.jl/pull/74). EDIT: There's a single branch, 1.5.3 and 1.6 simply point to a particular commit in that branch. 1.6 uses a relatively recent commit, but master contains a few commits which will only be in 1.7 at this point.

Thanks for offering your help. I think it's best to start from these two PRs. What's tricky is that as suggested at JuliaLang/julia#31395 (comment) I extracted all the StatsBase.jl history into the StatsBase branch of Statistics.jl, so that we don't lose it. But that means that we have to port all features we want in a single PR (otherwise we would duplicate history, which would be messy). Also unfortunately that branch isn't up to date with StatsBase now so we'll have to sync it again (but maybe once everything is ready to avoid doing work multiple times).

Regarding which features to port, I had written something here: JuliaLang/julia#27152 (comment) At least we should port things from the first section. Maybe also a few things from the second one, but we should be careful about reviewing APIs so that we don't commit to problematic ones.

[nalimilan]

#735 reexports functions from Statistics, including std.