consistency of std interface (Trac #1200)

[scipy-gitbot]

Original ticket http://projects.scipy.org/scipy/ticket/1200 on 2010-06-16 by trac user amcmorl, assigned to unknown.

scipy.stats.nanstd and numpy.std have different signatures, which should be made consistent:

scipy.stats.nanstd(x, axis=0, bias=False)

vs

numpy.std(a, axis=None, dtype=None, out=None, ddof=0)

Most important is changing the bias keyword to ddof.
The dtype parameter may not be relevant to nanstd since nans only exist in floats (what about complex?).

[scipy-gitbot]

Attachment added by @bsouthey on 2010-06-16: newstd.py

[scipy-gitbot]

@bsouthey wrote on 2010-06-16

Just a few explanations.

I created two new functions, one for the variance and one for the standard deviation (since it the square root of the variance).

The dtype argument is for the internal calculations and is only relevant when the dtype argument is float128. Numpy automatically converts the input to float64 so dtype is only used when for dtypes with higher precision (currently float128). HoweverSo I cheated by only converting the input to float128 when the dtype was float128.

I do not know how to implement the out argument although it can be achieved by converting the input to a masked array.

I also added a check for masked arrays so that if the input is a masked array so that the appropriate masked array function is used.

[scipy-gitbot]

@rgommers wrote on 2010-11-21

First on the signatures (of nanstd, but other nan functions as well): this can not be done in a backwards-compatible way, which is a pain. Both axis=0 vs axis=None and bias=False vs ddof=0 will change the default behavior. So we need to decide whether to clean this up or not, and if so, how. I attach a patch for the bias kw, this would be one way to do it.

On Bruce's patch: a nanvar function would be a good addition. But the nan* funcs do not handle masked arrays, neither should this one I think. That's what mstats is for. Furthermore the axis handling works only for 2-D it seems, look at other nan* funcs to see how it's done there (specifically _chk_asarray). The print statements at the end should be converted to tests.

[scipy-gitbot]

Attachment added by @rgommers on 2010-11-21: stats_biaskw.patch

[scipy-gitbot]

@rgommers wrote on 2011-08-19

Do we want to apply this patch for 0.10?

[scipy-gitbot]

@rgommers wrote on 2011-09-11

Tried to make this change, it's not pretty. I don't want to do this for 0.10, and preferably not at all. We should have a nanstd in numpy relatively soon.

[josef-pkt]

I don't think it's worth anymore to worry about this
nanmean, nanstd and nanvar have been merged into numpy
numpy/numpy#3534

We can deprecate and remove them from scipy.stats when the required numpy version is large enough.

Do we need an open issue for a deprecation (or FutureWarning) cycle and for delegating to numpy.nan in the meantime, if the numpy versions are available?

added milestone 0.14

[rgommers]

Should be OK to deprecate for 0.14.0. No hurry to do it now.

[rgommers]

Or 0.15.0 - didn't get around to this in time.