Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Test Failure in LLE on ubuntu 12.10 #1632

Closed
amueller opened this Issue · 13 comments

5 participants

Andreas Mueller Alexandre Gramfort Jake Vanderplas devinshields Kyle Kastner
Andreas Mueller
Owner

So my box in uni got upgraded and now I have this test failure:

======================================================================
FAIL: sklearn.manifold.tests.test_locally_linear.test_lle_simple_grid
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/local/lamueller/checkout/scikit-learn/sklearn/manifold/tests/test_locally_linear.py", line 68, in test_lle_simple_grid
    assert_less(np.linalg.norm(X_reembedded - clf.embedding_), tol)
AssertionError: 0.12820616722921346 not less than 0.1
    """Fail immediately, with the given message."""
>>  raise self.failureException('0.12820616722921346 not less than 0.1')

scipy 0.10.1, 32bit.

Alexandre Gramfort
Owner
Jake Vanderplas
Collaborator

My guess is this is due to ARPACK - it's been notoriously unstable between systems; so much so that scipy devs have disabled the 32-bit wrappers starting in 0.10.1 (see http://docs.scipy.org/doc/scipy-0.10.1/reference/release.0.10.1.html). If you're using an earlier scipy version on a 32-bit system, then that's probably the issue. Even in newer versions, I wouldn't be surprised if ARPACK is causing problems.

Andreas Mueller
Owner

So what is the fix? Less strict test? Less strict test if we have 32bit scipy (can we test that, maybe via numpy.distutils.system_info.platform_bits)?

Jake Vanderplas
Collaborator

Hard to say... ARPACK is so unpredictable. I suppose a less-strict test could work here, but there still might be errors on other systems out there.

Andreas Mueller
Owner

We do really strive for no errors, so I guess we should fix all that we know about. (for some value of "fix")

Andreas Mueller
Owner

Can't reproduce any more. @agramfort do you still see it? Not sure what changed in my box. If you do I guess I'll change the test a bit. Do you have the same numbers?

Alexandre Gramfort
Owner
Andreas Mueller
Owner

that is pretty far of... wait, your error is way smaller than expected. Is that right?

Alexandre Gramfort
Owner
devinshields

I ran nose on a vanilla EC2 64 bit Ubuntu 12.10 and received no errors. The full install and test log are available here:
https://gist.github.com/devinshields/5523616

Andreas Mueller
Owner

@davinshields the problem is 32bit related

devinshields

Sorry, here are the 32 bit results - the tests are still failing but in a different spot. A full dump is here:
https://gist.github.com/devinshields/5523760

FAIL: sklearn.tests.test_common.test_transformers

Traceback (most recent call last):
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/sklearn/tests/test_common.py", line 230, in test_transformers
"fit_transform not correct in %s" % Trans)
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 812, in assert_array_almost_equal
header=('Arrays are not almost equal to %d decimals' % decimal))
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 645, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 2 decimals
fit_transform not correct in
(mismatch 50.0%)
x: array([[ 0.97459066, -0.37275475],
[-0.97199043, 0.22191067],
[-0.97199043, 0.22191067],...
y: array([[ 9.74590662e-01, -5.79470839e-17],
[ -9.71990427e-01, -5.85363968e-17],
[ -9.71990427e-01, -5.85363968e-17],...


Ran 1598 tests in 107.531s

FAILED (SKIP=15, failures=1)

Andreas Mueller amueller added this to the 0.15.1 milestone
Kyle Kastner
Owner

Closing as duplicate - trying to organize all of the 32bit numerical stability bugs in one place, so we can squash them all :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.