# Make the orthogonal polynomial eval_* functions ufuncs. #339

Merged
merged 15 commits into from Dec 1, 2012

## Conversation

Projects
None yet
2 participants
Member

### ewmoore commented Oct 17, 2012

 As I proposed here and branching from Pauli's work in #333 and #334, I've moved the functions for evaluating orthogonal polynomials to cython and made them into ufuncs. At present there are no changes to how things are actually evaluated. Since this is based on #333 and #334, only ed50214 and on are of interest here. At present none of the polynomials can be evaluated for a complex number. This probably needs to be fixed since it removes an existing feature. I'll add this soon. Ticket 1435 still needs to be addressed, but this is easy, too (Need to change orthopoly1d.call). Some other polynomials can also be evaluated using recurrences as long as care is taken, (c.f. http://nalag.cs.kuleuven.be/papers/paul/report74/report74.ps.gz). This ought to be faster for small n at least, so I'd like to add it where it makes sense. I'm submitting this as something of a work in progress because wanted to get something up before I ended up just letting it rot on my machine.
Member

### ewmoore commented Oct 26, 2012

 So I'd like a little input before I spend more time here. I've started filling out various routines using the recurrances directly. However I see two possible issues here. We are now doing dispatch to different implementations based on the type of the array that is passed. This ends up meaning that for some functions you will possibly get a NaN or not depending on the type. For instance: ``````In [23]: s.eval_legendre(44000,np.linspace(-1,1,20),sig='ld->d') Out[23]: array([ 1.00000000e+00, 4.59024091e-03, -1.30797158e-03, 2.07979703e-03, 2.44349495e-03, -8.74737533e-04, -3.29821075e-04, 1.85696376e-03, -2.57296402e-03, -1.45712334e-04, -1.45712334e-04, -2.57296402e-03, 1.85696376e-03, -3.29821075e-04, -8.74737533e-04, 2.44349495e-03, 2.07979703e-03, -1.30797158e-03, 4.59024091e-03, 1.00000000e+00]) In [24]: s.eval_legendre(44000,np.linspace(-1,1,20),sig='dd->d') /home/ewm/pyenvs/scipy/bin/ipython:1: RuntimeWarning: overflow encountered in eval_legendre #!/home/ewm/pyenvs/scipy/bin/python /home/ewm/pyenvs/scipy/bin/ipython:1: RuntimeWarning: invalid value encountered in eval_legendre #!/home/ewm/pyenvs/scipy/bin/python Out[24]: array([ nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, -2.57296402e-03, 1.85696376e-03, -3.29821075e-04, -8.74737533e-04, 2.44349495e-03, 2.07979703e-03, -1.30797158e-03, 4.59024091e-03, 1.00000000e+00]) `````` The way generate_ufuncs.py currently works means that you cannot define a ld->d function and a dd->d function and have things work as expected. Either they end up being defined in an incorrect order (dd->d, ld->d, instead of ld->d, dd->d) or it selects the ld->d version for all of them and rounds the doubles to longs. Is this by design or does this just need to be extended? I'm also curious if I should just wait and see how the possible change from silently rounding doubles to longs to raising an error comes out? This was discussed in a recent pull request, although I'm not sure right off which one that was. Thanks, Eric
Owner

### pv commented Oct 26, 2012

 The dispatch mechanism indeed needs a tweak, so that the exact signatures (with "i>l" replacement) come first, and all up/downcast versions follow after that. This was the intent, but wasn't required by any of the existing functions, so the current behavior is incorrect. The outcome from the double -> long downcast issue is probably that we will simply not generate the "l>d" versions of the functions so that an error is raised (so that the error is determined by input type), or we add data value checks in the downcast loops (so that the error is raised by input value). The "i>l" downcast probably needs checking by input value. The issue is that on 64-bit systems it can be that i != l, with l being the main integer type, and this can cause major inconvenience if only i is accepted. The backwards compatibility for the previous unsafe functions is done by adding manually the unsafe versions to the lists. However, I don't see this issue overlapping very much with this PR. It might be interesting to see, however, how fast the recurrence-based evaluation is. The advantage of using the existing special function routines is that they tend to have asymptotic expansions which work also when the evaluation would require an absurd number of cycles in the recurrence. One alternative is of course to use the recurrences when they make sense, and fall back to the special functions if not.
Member

### ewmoore commented Oct 30, 2012

 I've investigated the speed of the recurrence for Legendre polynomials. I've looked at n between 0 and 10,000 for x vectors of 10, 100, 1000, 10,000 elements. The recurrence is almost always faster, except for small vectors at high order. Some plots: (curves labeled ld use the recurrence, and dd call out to hyp2f1) dropbox, where the image is hosted seems to be having issues with this one. This is the same data as the above plot where each curve has been normalized by the length of x.
Member

### ewmoore commented Oct 30, 2012

 Since Jacobi, Gegenbauer, and Chebyshev polynomials also result in calls to hyp2f1 they'll likely be faster using the recurrence directly as well. I'll check Laguerre and Hermite separately.

Owner

### pv commented Oct 31, 2012

 Hi, thanks for the results. Now that I think of it again, I fixed our hyp2f1 implementation at one point by adding recurrence evaluation (hyp2f1.c:hyp2f1ra), which is probably equivalent to a generalized ortho-polynomial recurrence. Importantly, there is no code path in hyp2f1 for asymptotic expansions in large orders, so it will never be faster than the recurrence. Which means that orthogonal polynomials with only recurrences can only improve the situation. Of course, adding asymptotic expansions to hyp2f1 would be possible, but this requires digging up literature, and there are more important things to fix in scipy.special...
Owner

### pv commented Nov 11, 2012

 Note that you can use now also fused types to get the complex-valued versions with the same amount of work. In the ufunc loop description, one can then specify `somefunc[double]: d->d, somefunc[double complex]: D->D`
Owner

### pv commented Nov 25, 2012

 Ok, this looks good. (Sorry for the delay, apparently Github doesn't send notifications for only pushes, but only for comments).

### ewmoore and others added some commits Oct 17, 2012

``` ENH: make orthogonal poly eval routines ufuncs ```
``` d35c54a ```
``` BUG: special: orthopoly1d.__call__ shouldn't type check ```
`This fixes trac ticket 1435, depends on having eval* be ufuncs.`
``` a27a690 ```
``` BUG: special: orthopoly1d.__call__ should type check for np.poly1d ```
`objects.`
``` 01d0939 ```
``` ENH: special: eval_* restore evaluating for complex args ```
```This covers everything but hermite, see a later commit.  The order these
functions are listed in generate_ufuncs.py probably needs to be fixed
later too.```
``` f30d423 ```
``` ENH: special: add path using recurances to eval_* ```
`This commit handles everything but hermite.`
``` 7854542 ```
``` ENH: special: use fused types in orthogonal_eval ```
``` e8e349d ```
``` BUG: special: deal with spaces in fused type names e.g. 'double complex' ```
``` 1df0e9e ```
``` GEN: special: regenerate files ```
``` a1a2658 ```
``` BUG: special/orthogonal_eval: fix bug in eval_sh_jaboci_l ```
``` a7b5720 ```
Owner

### pv commented Nov 25, 2012

 Rebased branch here: https://github.com/pv/scipy-work/tree/ortho_eval I also converted the complex loops to use fused types. To work on that: ``````git remote add pv git://github.com/pv/scipy-work.git git fetch pv git checkout ortho_eval git branch ortho_eval_backup ortho_eval git reset --hard pv/ortho_eval # NB. resets also working tree `````` Apparently, the long-loops were not run before due to the loop selection issues in `generate_ufuncs.py`. They are now after the rebase, and the tests currently seem to indicate that the Jacobi and Gegenbauer polynomial recurrences apparently give wrong results. Also, some new tests need to be added to compare the double and long loops to each other.
Member

### ewmoore commented Nov 28, 2012

 Ok, I've pulled down your changes and have started looking at the failures. I'm using forms of the recurrences given here: https://lirias.kuleuven.be/handle/123456789/13160 . Which may or may not be necessary. One of things they do is to work with a rescaled recurrence. That rescaling for the Gegenbauer polynomials rescales the value at x = 1.0 to 1.0, using C(n, a, 1.0) = binom(n+2a-1, n). However, this requires that the binomial coefficient, binom(n,k) be computed for non-integer values of n. This doesn't seem like a big deal, but it gets tricky. We can extend the binomial coefficient using either of the gamma or beta function, but we are calculating it using gammaln and exp, which is giving us the wrong sign when the binomial coefficient should be negative. Consider: ```In [1]: import numpy as np; import scipy.special as s; In [2]: n = 0.264 In [3]: k = 2 In [4]: s.binom(n,k) Out[4]: 0.097152000000000016 In [5]: np.exp(s.gammaln(n+1) - s.gammaln(k+1) - s.gammaln(n-k+1)) Out[5]: 0.097152000000000016 In [6]: s.gamma(n+1) / ( s.gamma(k+1) * s.gamma(n-k+1) ) Out[6]: -0.097152000000000002 In [7]: 1.0 / ((n+1) * s.beta(n-k+1, k+1)) Out[7]: -0.09715200000000003``` Computing the binomal coefficient directly with gamma or beta fixes the Gegenbauer failure, I'll push something tomorrow.. It may well be something similar for Jacobi, I'll have to see. It seems to me that the current binom implementation gives the wrong answer at times.
Owner

### pv commented Nov 28, 2012

 In [5]: np.exp(s.gammaln(n+1) - s.gammaln(k+1) - s.gammaln(n-k+1)) Out[5]: 0.097152000000000016 In [6]: s.gamma(n+1) / ( s.gamma(k+1) * s.gamma(n-k+1) ) Out[6]: -0.097152000000000002 Indeed, gammaln gives the logarithm of the absolute value. We'd probably want an additional function in scipy.special giving the sign of the Gamma function. On the real axis, ``````sgn Gamma(x) = (-1)**(floor(x)) , x < 0 sgn Gamma(x) = 1, x > 0 `````` The sign at non-positive integers is undefined, but I think it should be chosen as a finite number rather than nan because we want ``````1 / Gamma(x) = gammasgn(x) * exp(-gammaln(x)) `````` be an entire function. gammasgn(-n) = 1 is probably not a bad choice, and gammasgn(-n) = 0 would have the advantage that for it, gammasgn(-n) * exp(gammaln(-n)) = nan rather than inf. I'd avoid the expression in terms of plain gamma functions --- that will start spitting out nans for large `n` even when the binomial coefficient is not large. The expression with the Beta function is probably OK, although may need an additional power series expansion (very) close to the limit k->0 (or at least dealing with the point k=0). Related: we'd also need a new implementation of the Pochhammer symbol, Pochhammer(a, n) = Gamma(a + n) / Gamma(a). Thanks,

### ewmoore added some commits Nov 29, 2012

``` ENH: special: add a gammasgn function ```
``` d2a362c ```
``` BUG: special: get the sign of binom right ```
`Also add a test.`
``` db1affd ```
``` BUG: special: use eval_gegenbauer as eval_func in gegegbauer ```
``` 62ea0b3 ```
Member

### ewmoore commented Nov 29, 2012

 I can't seem to push my new changes. (scipy)ewm@hollow:/scipy\$ git push Username for 'https://github.com': ewmoore Password for 'https://ewmoore@github.com': To https://github.com/ewmoore/scipy.git ! [rejected] ortho_eval -> ortho_eval (non-fast-forward) error: failed to push some refs to 'https://github.com/ewmoore/scipy.git' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. (>scipy)ewm@hollow:/scipy\$ Any ideas?
Owner

### pv commented Nov 29, 2012

 `git push -f` -- rebased commits are not on top of the old ones, so it doesn't let you push them without confirmation.

### ewmoore added some commits Nov 30, 2012

``` BUG: special: correct recurrance in eval_jacobi_l ```
`all current tests now pass.`
``` cc0cbce ```
``` TST: special: test eval_* ld->d against dd->d ```
``` e0e9268 ```
Member

### ewmoore commented Nov 30, 2012

 Okay, I've fixed up the issues with eval_gegenbauer and eval_jacobi, added a gammasgn function with test, fixed the bug in binom, add a test and add a test between the ld->d and dd->d version. The only thing I know of that might still be outstanding here is that we don't have a dd->d version for either form of the Hermite polynomials. There are a bunch of relationships that might work, but at this point we haven't lost any abilities we had before I started.
``` GEN: special: regenerate files ```
``` e1ae0a0 ```
Member

### ewmoore commented Nov 30, 2012

 Oh and I see that lgam actually provides the sign in cephes, but it does so rather unpleasantly by placing it in a global variable named sgngam. This isn't AFAICT exposed anywhere. I don't think that we could actually use sgngam in binom without moving that to cephes. I also don't really see a backward compatible way to add the sign to the exposed gammaln function.

### pv added a commit that referenced this pull request Dec 1, 2012

``` Merge pull request #339 from ewmoore/ortho_eval ```
```ENH: special: Make the orthogonal polynomial eval_* functions ufuncs

Convert the orthogonal polynomial eval_* functions to real ufuncs, and
implement their evaluation via recurrences where applicable.  This makes e.g.
their out= arguments work properly, and makes the simple cases faster.

Also add new gammasgn() and binom() functions.```
``` 9b4c5eb ```

Owner

### pv commented Dec 1, 2012

 Thanks, merged. The general Hermite polynomials can go in a separate PR if needed. I opted in the end to go for the beta function relation in binom(), because the case n >> k needs special handling to avoid loss of precision. Unfortunately, this isn't yet implemented in the beta() function either, but that's a separate bug.
Owner

### pv commented Dec 1, 2012

 Hmm, lots of failures turn up on mingw-windows build: http://pav.iki.fi/tmp/buildbot/builders/scipy-wine-py2.7/builds/54/steps/test/logs/stdio.html Not immediately clear what's going on --- some routines seem to be returning nans. Some syntax unsupported by earlier Python versions was also present: http://pav.iki.fi/tmp/buildbot/builders/scipy-py2.5/builds/69/steps/test/logs/stdio.html

Merged

Owner

### pv commented Dec 2, 2012

 facd638 fixed the nan issues.

Closed

Closed

### rgommers referenced this pull request Oct 4, 2013

Closed

#### test_orthogonal_eval.TestRecurrence issue with MinGW build #2961

to join this conversation on GitHub. Already have an account? Sign in to comment