New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHORE hyp2F1 #500

Closed
samuelstjean opened this Issue Dec 12, 2014 · 7 comments

Comments

Projects
None yet
3 participants
@samuelstjean
Contributor

samuelstjean commented Dec 12, 2014

Just saw this, but while we are at it, scipy.special.hyp2F1 is also unstable (scipy/scipy#1561, scipy/scipy#3479), so the conversation in #463 should apply here if it's properly converted.

The only cases where those hypPFQ functions are used is in shore and my to-be-coming stuff on noise estimation acording to a quick git grep.

@maurozucchelli

This comment has been minimized.

Contributor

maurozucchelli commented Dec 12, 2014

I get a bit lost in the conversation on the other PR. Do you have an alternative to scipy hyp2f1 for the SHORE?

@arokem

This comment has been minimized.

Member

arokem commented Dec 12, 2014

Could you please provide a PR with a test-case that fails for the SHORE implementation? That would help focus the discussion.

@samuelstjean

This comment has been minimized.

Contributor

samuelstjean commented Dec 12, 2014

Yes ,two of them, one is using the gnu gsl (which is gpl, hence the debate on the mailing list), the other is using code from mpmath. @matthew-brett already converted most of it to pure python, but the mpmath version is known to be unusably slow sadly (like, 2 minutes for unstable scipy vs 6 hours from my experience with hyp1f1).

The second option would then be to use that mpmath converted code and port it to cython so it becomes usable. There is a gist in the aformentionned discussion with the code and my equally crude cython port.

@samuelstjean

This comment has been minimized.

Contributor

samuelstjean commented Dec 12, 2014

Well, it seems to fail in some specific cases, and give misleading results in others. The usage of hyp2f1 in shore seems simple enough (always posiitive, small numbers so far), so it might not trigger any issue.

Just be aware that it's sister function function does not works in same interesting cases for the project sadly, so if we ever re-implement them, this also comes as free. There might be any other use case where hyp2f1 is useful and fall into problematic ranges, but so far I have no easy example to provide.

@arokem

This comment has been minimized.

Member

arokem commented Dec 13, 2014

Do I understand correctly that these cases are irrelevant to SHORE? Can we
close this?

Otherwise, if I am misunderstanding, could you please write down (in code,
as a test) a few cases in which this fails and/or gives misleading results?
We need these as a starting point to fix this. Otherwise, this is just too
theoretical for me.

BTW, this seems like a useful a potentially useful reference implementation
of some of this stuff [link downloads zip file!]:

http://people.maths.ox.ac.uk/~porterm/research/hypergeometricpackage.zip

On Fri, Dec 12, 2014 at 9:35 AM, Samuel St-Jean notifications@github.com
wrote:

Well, it seems to fail in some specific cases, and give misleading results
in others. The usage of hyp2f1 in shore seems simple enough (always
posiitive, small numbers so far), so it might not trigger any issue.

Just be aware that it's sister function function does not works in same
interesting cases for the project sadly, so if we ever re-implement them,
this also comes as free. There might be any other use case where hyp2f1 is
useful and fall into problematic ranges, but so far I have no easy example
to provide.


Reply to this email directly or view it on GitHub
#500 (comment).

@samuelstjean

This comment has been minimized.

Contributor

samuelstjean commented Dec 13, 2014

Don't worry, it seems more like a hypothetical problem since the SHORE
model doesn't seems to fall into the identified problematic range.
Problem is, that might not be the case someday who knows, but we'll see
if/when that happens I guess. Even fi it doesn't give nan/inf,
identifying bogus values is hard anyway, so if we ever find another
reliable implementation, we shoudl use that instead.
Le 2014-12-13 14:08, Ariel Rokem a écrit :

Do I understand correctly that these cases are irrelevant to SHORE?
Can we
close this?

Otherwise, if I am misunderstanding, could you please write down (in
code,
as a test) a few cases in which this fails and/or gives misleading
results?
We need these as a starting point to fix this. Otherwise, this is just
too
theoretical for me.

BTW, this seems like a useful a potentially useful reference
implementation
of some of this stuff [link downloads zip file!]:

http://people.maths.ox.ac.uk/~porterm/research/hypergeometricpackage.zip

On Fri, Dec 12, 2014 at 9:35 AM, Samuel St-Jean
notifications@github.com
wrote:

Well, it seems to fail in some specific cases, and give misleading
results
in others. The usage of hyp2f1 in shore seems simple enough (always
posiitive, small numbers so far), so it might not trigger any issue.

Just be aware that it's sister function function does not works in same
interesting cases for the project sadly, so if we ever re-implement
them,
this also comes as free. There might be any other use case where
hyp2f1 is
useful and fall into problematic ranges, but so far I have no easy
example
to provide.


Reply to this email directly or view it on GitHub
#500 (comment).


Reply to this email directly or view it on GitHub
#500 (comment).

@arokem

This comment has been minimized.

Member

arokem commented Dec 13, 2014

OK - closing this for now.

@arokem arokem closed this Dec 13, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment