Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Test Failure in LLE on ubuntu 12.10 #1632

Closed
amueller opened this Issue · 13 comments

5 participants

@amueller
Owner

So my box in uni got upgraded and now I have this test failure:

======================================================================
FAIL: sklearn.manifold.tests.test_locally_linear.test_lle_simple_grid
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/local/lamueller/checkout/scikit-learn/sklearn/manifold/tests/test_locally_linear.py", line 68, in test_lle_simple_grid
    assert_less(np.linalg.norm(X_reembedded - clf.embedding_), tol)
AssertionError: 0.12820616722921346 not less than 0.1
    """Fail immediately, with the given message."""
>>  raise self.failureException('0.12820616722921346 not less than 0.1')

scipy 0.10.1, 32bit.

@agramfort
Owner
@jakevdp
Collaborator

My guess is this is due to ARPACK - it's been notoriously unstable between systems; so much so that scipy devs have disabled the 32-bit wrappers starting in 0.10.1 (see http://docs.scipy.org/doc/scipy-0.10.1/reference/release.0.10.1.html). If you're using an earlier scipy version on a 32-bit system, then that's probably the issue. Even in newer versions, I wouldn't be surprised if ARPACK is causing problems.

@amueller
Owner

So what is the fix? Less strict test? Less strict test if we have 32bit scipy (can we test that, maybe via numpy.distutils.system_info.platform_bits)?

@jakevdp
Collaborator

Hard to say... ARPACK is so unpredictable. I suppose a less-strict test could work here, but there still might be errors on other systems out there.

@amueller
Owner

We do really strive for no errors, so I guess we should fix all that we know about. (for some value of "fix")

@amueller
Owner

Can't reproduce any more. @agramfort do you still see it? Not sure what changed in my box. If you do I guess I'll change the test a bit. Do you have the same numbers?

@agramfort
Owner
@amueller
Owner

that is pretty far of... wait, your error is way smaller than expected. Is that right?

@agramfort
Owner
@devinshields

I ran nose on a vanilla EC2 64 bit Ubuntu 12.10 and received no errors. The full install and test log are available here:
https://gist.github.com/devinshields/5523616

@amueller
Owner

@davinshields the problem is 32bit related

@devinshields

Sorry, here are the 32 bit results - the tests are still failing but in a different spot. A full dump is here:
https://gist.github.com/devinshields/5523760

FAIL: sklearn.tests.test_common.test_transformers

Traceback (most recent call last):
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/sklearn/tests/test_common.py", line 230, in test_transformers
"fit_transform not correct in %s" % Trans)
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 812, in assert_array_almost_equal
header=('Arrays are not almost equal to %d decimals' % decimal))
File "/home/ubuntu/venv/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 645, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 2 decimals
fit_transform not correct in
(mismatch 50.0%)
x: array([[ 0.97459066, -0.37275475],
[-0.97199043, 0.22191067],
[-0.97199043, 0.22191067],...
y: array([[ 9.74590662e-01, -5.79470839e-17],
[ -9.71990427e-01, -5.85363968e-17],
[ -9.71990427e-01, -5.85363968e-17],...


Ran 1598 tests in 107.531s

FAILED (SKIP=15, failures=1)

@amueller amueller added this to the 0.15.1 milestone
@kastnerkyle
Owner

Closing as duplicate - trying to organize all of the 32bit numerical stability bugs in one place, so we can squash them all :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.