a bit of maintanence work in stats #3243

Merged
merged 7 commits into from Jan 31, 2014

Projects

None yet

6 participants

@ev-br
SciPy member

closes gh-2741 and gh-2871
also remove a couple of unused and duplicate attributes of distributions.

@josef-pkt josef-pkt and 1 other commented on an outdated diff Jan 26, 2014
scipy/stats/mstats_basic.py
@@ -2006,7 +2006,7 @@ def f_oneway(*args):
msb = ssbg/float(dfbg)
msw = sswg/float(dfwg)
f = msb/msw
- prob = stats.fprob(dfbg,dfwg,f)
+ prob = special.fdtrc(dfbg, dfwg, f)
@josef-pkt
josef-pkt Jan 26, 2014

using stats.f.sf would be more informative

@ev-br
ev-br Jan 26, 2014

OK, have added an explicit comment in the last commit (+ similar in mstats).

@josef-pkt josef-pkt commented on an outdated diff Jan 26, 2014
scipy/stats/stats.py
@@ -2375,7 +2375,7 @@ def f_oneway(*args):
msb = ssbn / float(dfbn)
msw = sswn / float(dfwn)
f = msb / msw
- prob = fprob(dfbn, dfwn, f)
+ prob = special.fdtrc(dfbn, dfwn, f)
@josef-pkt
josef-pkt Jan 26, 2014

using stats.f.sf would be more informative

@josef-pkt josef-pkt and 1 other commented on an outdated diff Jan 26, 2014
scipy/stats/stats.py
@@ -3873,7 +3873,7 @@ def ks_2samp(data1, data2):
# Note: d absolute not signed distance
en = np.sqrt(n1*n2/float(n1+n2))
try:
- prob = ksprob((en+0.12+0.11/en)*d)
+ prob = special.kolmogorov((en + 0.12 + 0.11 / en) * d)
@josef-pkt
josef-pkt Jan 26, 2014

I'm not sure if the stats distribution is more informative.
(I don't remember why there is a try ... except. with a blank except)
might be a separate issue to understand this, not this PR.

@ev-br
ev-br Jan 26, 2014

I really don't think stats.kstwobign.sf (not sure how many typos I just made here) is any better than special.kolmogorov (OK, being used to Russian names I might have an advantage here).

@josef-pkt
josef-pkt Jan 26, 2014

kstwobign == ks_two_big_n == kolmogorov-smirnov for two sample test with a large number of observations
very informative
(IIRC it's the distribution of the max of a Brownian Bridge, or maybe that's the smirnov one :) maxbb distribution.

I never know which is kolmogorov and which is smirnov.

@ev-br
ev-br Jan 26, 2014

Secret knowledge indeed. Omit your underscores thou shalt not.
Do you want me to use kstwobign.sf or add an explanatory comment or leave this as is?

@josef-pkt
josef-pkt Jan 26, 2014

I prefer kstwobign.
see https://github.com/ev-br/scipy/blob/distr_cleanup/scipy/stats/stats.py#L3424 where I worked on different approximations for ks_test

@ev-br
ev-br Jan 27, 2014

updated in the last commit

@josef-pkt
SciPy member

looks good to me

@coveralls

Coverage Status

Coverage remained the same when pulling 63f5807 on ev-br:distr_cleanup into 8ad48cd on scipy:master.

@coveralls

Coverage Status

Coverage remained the same when pulling 3a92e44 on ev-br:distr_cleanup into 8ad48cd on scipy:master.

@coveralls

Coverage Status

Coverage remained the same when pulling 0e29858 on ev-br:distr_cleanup into 8ad48cd on scipy:master.

@rgommers rgommers merged commit 64fbe99 into scipy:master Jan 31, 2014

1 check passed

Details default The Travis CI build passed
@rgommers
SciPy member

LGTM, thanks @ev-br, @josef-pkt

@ev-br ev-br deleted the ev-br:distr_cleanup branch Jan 31, 2014
@ev-br ev-br referenced this pull request Feb 14, 2014
Closed

vecentropy.nin is wrong #2871

@yarikoptic

or in other words this broke compatibility without any warning/deprecation cycle... would be nice if API would not get such maintenance fixes in the future (could have wrapped into a property with a DeprecationWarning being spitted out). Also docs still need adjustment

doc/source/tutorial/stats.rst-``dir(norm)``.  As it turns out, some of the methods are private
doc/source/tutorial/stats.rst-methods although they are not named as such (their name does not start
doc/source/tutorial/stats.rst:with a leading underscore), for example ``veccdf``, are only available
doc/source/tutorial/stats.rst-for internal calculation.
@rgommers
SciPy member

Hmm, veccdf, vecfunc and vec_generic_moment should indeed have been deprecated. No idea why I overlooked that when merging this PR. Those methods probably didn't work as expected for some distributions, but that doesn't really matter.

Guess it doesn't help much to put them back in 0.14.1 as deprecated methods, but maybe we should do that anyway.

@ev-br
SciPy member

I still think all these were private implementation details of going from computational _XXX methods to user-callable XXX methods, but obviously I'm on a trigger-happy side when it comes to deprecations.

FWIW, I think adding them back in 0.14.1 does not make much sense. To have an alternative, I'd offer to help fixing the code which this change broke downstream. @yarikoptic care to share a link?

@pv
SciPy member
pv commented May 17, 2014

Adding them back might make sense, as 0.14.0 has not been out for a long time yet, and for example Linux distributions always lag behind.

@rgommers rgommers added a commit to rgommers/scipy that referenced this pull request May 26, 2014
@rgommers rgommers DEP: put back veccdf/vecfunc/vec_generic_momemt in stats.distributions.
Removed erroneously in gh-3243 for 0.14.0.  These methods were always
private ones but not marked as such, and as the comments on gh-3243 show
they were used. Hence now deprecated.
556220b
@rgommers
SciPy member

PR to put them back: gh-3685.

@rgommers rgommers added a commit that referenced this pull request Jul 5, 2014
@rgommers rgommers DEP: put back veccdf/vecfunc/vec_generic_momemt in stats.distributions.
Removed erroneously in gh-3243 for 0.14.0.  These methods were always
private ones but not marked as such, and as the comments on gh-3243 show
they were used. Hence now deprecated.

(cherry picked from commit 556220b)
b5db4cc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment