Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fixes gh-7491, pdf at x=0 of fisk/burr/burr12/f distributions. #10381

Merged
merged 1 commit into from
Jul 1, 2019

Conversation

pvanmulbregt
Copy link
Contributor

Replaces gh-9983 (rebase had trouble merging with gh-10154 --- this PR applies the same modifications manually to current master.)

Enable computation of pdf, logpdf at left-hand end points of fisk, burr,
burr12 and f distrbutions. Fixes gh-7491 and half of gh-9544.
Incorporate gh-9978 from @rlucas7 to address gh-9544.
Updated docstring of stats.burr distribution.
Added tutorial for stats.burr12.
Removed an ignored implementation of fisk._stats().
In stats.burr.{pdf,logpdf} replaced two separate long code blocks with a simpler _lazywhere.
Explicitly handle c<=n in stats.burr._munp().
Added consistency checking of pdf and logpdf to stats.test_continuous_basic.py.
Added a filter for stats.recipinvgauss in check_pdf_logpdf().
Added any finite distribution endpoints to list of x-values checked in
{check_pdf_logpdf, check_sf_logsf, check_cdf_logcdf}.
Added another test case to test_burr_stats_9544().

Enable computation of pdf, logpdf at left-hand end points of fisk, burr,
burr12 and f distrbutions.  Fixes scipygh-7491 and half of scipygh-9544.
Incorporate scipygh-9978 from rlucas7 to address scipygh-9544.
Updated docstring of burr distribution.
Added tutorial for stats.burr12.
Removed an ignored implementation of fisk._stats().
In stats.burr.{pdf,logpdf} replaced two separate long code blocks with
a simpler _lazywhere.
Explicitly handle c<=n in stats.burr._munp().
Added checking of pdf-logpdf consistency to stats.test_continuous_basic.py.
Added a filter for stats.recipinvgauss in check_pdf_logpdf().
Added any finite distribution endpoints to list of x-values checked in
{check_pdf_logpdf, check_sf_logsf, check_cdf_logcdf}.
Added another test case to test_burr_stats_9544().
f2 = lambda x_, c_, d_: (c_ * d_ * (x_ ** (-c_ - 1.0)) /
((1 + x_ ** (-c_)) ** (d_ + 1.0))))
if output.ndim == 0:
return output[()]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed here, since this squeezing is done at the rv_generic.pdf anyway. Can be done as a follow-up though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all calls to ._pdf originate from inside .pdf(). rv_continuous._cdf_single() calls integrate.quad(self._pdf, ...).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many other distribution do not have this clause. Is it a bug in them all? Or is it a bug in _cdf_single?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

integrate.quad passes in scalars to self._pdf and expects a scalar back.
burr._pdf(self, x, c, d) has a conditional. It does this by invoking _lazywhere(...), and _lazywhere always returns an array, even if all its inputs are scalars. Hence the squeeze.

The _pdf of most distributions makes calls to numpy or scipy ufuncs which return a scalar if given scalars, so this issue doesn't affect them.

+ sc.xlogy(-c_ - 1, x_)
- sc.xlog1py(d_+1, x_**(-c_))))
if output.ndim == 0:
return output[()]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

@ev-br ev-br merged commit 5799067 into scipy:master Jul 1, 2019
@ev-br
Copy link
Member

ev-br commented Jul 1, 2019

Thanks @pvanmulbregt, @rlucas7, LGTM modulo nitpicks which can be addressed in a separate cleanup. Merged.

@ev-br ev-br added this to the 1.4.0 milestone Jul 1, 2019
@larsoner larsoner mentioned this pull request Jul 8, 2019
3 tasks
@pvanmulbregt pvanmulbregt deleted the fiskburr2 branch December 29, 2019 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Several scipy.stats distributions (fisk, burr, burr12, f) return incorrect pdf at left endpoint
2 participants