upgrade numpy/scipy to 1.8.1 and 0.14

[kiwifb]

Upgrade numpy/scipy to latest versions. scipy 0.14 has finally ditched oldnumeric which means it won't generate warnings all over the place. A few doctests will have to be updated.

Package sources:

• https://pypi.python.org/packages/source/n/numpy/numpy-1.8.1.tar.gz
• https://pypi.python.org/packages/source/s/scipy/scipy-0.14.0.tar.gz

Component: packages: standard

Author: François Bissey

Branch/Commit: 5a16124

Reviewer: Nathan Dunfield

Issue created by migration from https://trac.sagemath.org/ticket/16299

[kiwifb]

Branch: numpy181

[kiwifb]

Commit: 62c44ee

[kiwifb]

comment:4

OK so it

[kiwifb]

Changed branch from numpy181 to u/fbissey/numpy181

[kiwifb]

comment:5

Woopsie. Anyway I have updated the checksums and SPKG.txt for numpy and scipy. No changes where necessary to the packages themselves as far as I could tell. I fixed three doctests:

• src/sage/functions/exp_integral.py: the last digit of a test changed, I put "..." since it is probably a little unstable.
• src/sage/numerical/optimize.py: also a numerical precision thing. In this case we ask for a value within 0.0001 tolerance but tests the answer to 16 decimal places. No surprise when we have changes behind the 5th place.
• src/sage/modules/vector_double_dense.pyx: a doctest warning has changed code.

[kiwifb]

Author: François Bissey

[NathanDunfield]

comment:7

I tried out this branch and the new packages with Sage 6.2 on two systems: OX Mavericks and RHEL 6.3 (gcc 4.4). Both numpy and scipy installed cleanly and the three modified doctests passed on both systems. Other changes in the patch look fine, so setting status to "positive review".

[NathanDunfield]

Reviewer: Nathan Dunfield

[vbraun]

comment:9

Breaks doctests on OSX (10.6 and 10.9):

sage -t --long src/sage/tests/french_book/calculus_doctest.py
**********************************************************************
File "src/sage/tests/french_book/calculus_doctest.py", line 216, in sage.tests.french_book.calculus_doctest
Failed example:
    find_root(expr, 0.1, pi)
Expected:
    2.0943951023931957
Got:
    2.0943951023931953

[kiwifb]

comment:10

OK, will amend that. Any other failures I missed?

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5a16124

Fix doctest failing on OS X on the last digit after upgarding numpy/scipy to 1.8.1/0.14.0

[sagetrac-git]

Changed commit from 62c44ee to 5a16124

[kiwifb]

comment:12

OK fixed the doctest and pushed. Ready for another round of review.

[vbraun]

comment:13

I don't see any reason why this should give a different result on OSX. Do you?

[kiwifb]

comment:14

Not sure. We are talking about a difference in the 4E-16 which is in the relative tolerance of the function and well within its own tolerance.

def find_root(f, a, b, xtol=10e-13, rtol=4.5e-16, maxiter=100, full_output=False):

But I would have been more ready to accept a difference based on CPU arch rather than OS on similar CPU.

Looking at scipy the underlying c code hasn't changed since 2008 but higher up tolerance checking has been enforced in 0.14rc2 (scipy/optimize/zeros.py) that's the only thing I can see having an impact.

[NathanDunfield]

comment:15

I did a little testing, and definitely the issue is a result of the scipy upgrade and not the numpy one. It also shows up on OS X version 10.8 as well.

My experience is that the last digit in numerics like this can be remarkably sensitive to the OS/CPU/compiler version/phase of the moon. I once had two Ubuntu 12.04 virtual machines running on the same physical hardware where the last digit differed like this; one had a 32bit kernel and the other a 64bit one, which you wouldn't think would make a difference...

I think this sort of thing is to be expected, and it certainly seems harmless enough in this instance. So I've changed the status back to "positive review".

[kiwifb]

comment:16

Replying to @NathanDunfield:

I did a little testing, and definitely the issue is a result of the scipy upgrade and not the numpy one. It also shows up on OS X version 10.8 as well.

My experience is that the last digit in numerics like this can be remarkably sensitive to the OS/CPU/compiler version/phase of the moon. I once had two Ubuntu 12.04 virtual machines running on the same physical hardware where the last digit differed like this; one had a 32bit kernel and the other a 64bit one, which you wouldn't think would make a difference...

Oh, yes I would. But then I have seen tickets where that happens in the last 6 years.

I think this sort of thing is to be expected, and it certainly seems harmless enough in this instance. So I've changed the status back to "positive review".

It is technically harmless it is however a curiosity because the change is sudden. If I had time I would check other versions of scipy to narrow down when the change happened and try to understand why.

[kiwifb]

comment:17

In the end I spent the time checking scipy/scipy@8edf038 and scipy/scipy@0ee2411 are responsible. I didn't try to pin it on one or the other. So it looks like we approch the root slightly differently between OS X and linux. My guess is that without the enforcement of tolerance OS X would go through one (or) more iteration(s) and end up with the same value as Linux.

[vbraun]

Changed branch from u/fbissey/numpy181 to 5a16124