Roadmap

[LukeMathWalker]

In terms of functionality, the mid-term end goal is to achieve feature parity with the statistics routine in numpy (here) and Julia StatsBase (here).

For version 0.2.0:

• Order statistics:
  • partialord version for quantiles (using MaybeNan?);
  • optimized computations of multiple quantiles if requested all at once;
  • argmin / argmax;
• Summary statistics:
  • harmonic mean;
  • geometric mean;
  • higher order central moments;
  • standardized moments (they include kurtosis and skewness);
• Histograms:
  • Fix error handling (#16)
  • merge method;
• Entropy:
  • Feature parity with StatsBase.jl

For version 0.1.0:

• max / nanmax (@jturner314)
• min / nanmin (@jturner314)
• quantile / nanquantile (it includes percentile / nanpercentile as a special case) (@LukeMathWalker & @jturner314)
• correlation-methods:
  • cov (@LukeMathWalker) - One last fix to be made (#3) [On hold for now]
  • corrcoef (@LukeMathWalker - #5)
• histogram-methods (@LukeMathWalker - #9)

[jturner314]

With respect to mean, average, std: var is implemented in the main ndarray crate - would it make sense to port it here?

I think it makes sense for ndarray-stats to provide *_skipnan variants (or whatever you want to call them) of those methods. However, it would make sense to add std_axis to ndarray since ndarray already has var_axis.

For methods that are already in ndarray, we could duplicate these methods as a trait in ndarray-stats for people who want to write generic code (where the implementations just call the instance methods). I'm ambivalent on this.

I'll slowly start working on this next week and then I'll get serious the week afterwards. Could you please give me commit/PR permissions to the repository @jturner314?

Okay, that sounds good. I've given you push access. Alternatively, if you'd like to have your repo be the main one instead of this one, that would be fine with me.

[LukeMathWalker]

Once #9 gets merged I think we are in a good position to officially release version 0.1.0 on crates.io - what do you think? @jturner314

[jturner314]

I agree.

By the way, I recently came across Julia's StatsBase.jl library. It's a good source of ideas in addition to NumPy/SciPy.

[LukeMathWalker]

Added a bunch of tests to #9 and merged 🎉 It feels like ages since I started to work on it 😅 Your contribution was extremely helpful to get it in the shape it is right now, thanks a lot @jturner314!

What do we need to do in order to release on crates.io?
I am going to open a small PR to add crate-level documentation - a couple of lines, nothing major.

[jturner314]

Yay! 🎉 That was a big job; great work.

What do we need to do in order to release on crates.io?

Ideally, we'd eliminate the [patch.crates-io] section from the Cargo.toml before we can release on crates.io. (This might even be required, I'm not sure.) #11 removed the patch for noisy_float, but a new version of ndarray will need to be released for us to remove its patch. It would be nice to merge a couple more ndarray PRs before release; I'll take a look.

It would also be good to merge #12 and #13 before releasing.

[LukeMathWalker]

Merged #12 and #13 - looking around it seems we can publish with [patch.crates-io] section in Cargo.toml, but I agree it is much nicer to point to ndarray 0.12.1 as a dependency instead of a revision on master.

Let's wait for that release and then we are good to go.

[jturner314]

ndarray-stats 0.1.0 is now on crates.io. 🎉 Thanks for all your hard work @LukeMathWalker!

[LukeMathWalker]

💯 💯 I think it's safe to say it would have never got there without your help 😛
I'll drop a post on r/rust as well 👍

[LukeMathWalker]

I have drafted a tentative roadmap with the features I'd like to add in the next release - please edit it with your comments and suggestions @jturner314

[jturner314]

The roadmap looks good to me. I'm not familiar with the applications of higher order central moments (I'd usually use a histogram instead), but I don't mind adding them if people find them useful.

By the way, I invited you as an owner for the ndarray-stats crate, but I just realized that crates.io may not have sent the invitation if you haven't logged in before. Please let me know if you need me to re-send it.

[LukeMathWalker]

Somehow I didn't receive an email notification, but the invite was on my dashboard - accepted it!

The main objective in that area is getting kurtosis and skewness, and given the kind of computation required to achieve that it makes sense to also roll out higher order central moments I'd say :)