test_spectral_clustering_sparse fail on OS X 10.8.2 #2472

Closed
chyikwei opened this Issue Sep 22, 2013 · 12 comments

Comments

Projects
None yet
4 participants
Contributor

chyikwei commented Sep 22, 2013

I got this error on sklearn.cluster.tests.test_spectral.test_spectral_clustering_sparse
Any one knows what might be wrong?

OS: Max OS X 10.8.2
numpy: 1.7.1
scipy: 0.11.0


======================================================================
FAIL: sklearn.cluster.tests.test_spectral.test_spectral_clustering_sparse
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/chyikwei/getglue-python-webservice/virtual/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/chyikwei/scikit-learn/sklearn/cluster/tests/test_spectral.py", line 155, in test_spectral_clustering_sparse
    assert_greater(np.mean(labels == [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]), .89)
AssertionError: 0.59999999999999998 not greater than 0.89
    """Fail immediately, with the given message."""
>>  raise self.failureException('0.59999999999999998 not greater than 0.89')
Owner

ogrisel commented Sep 22, 2013

I am running a similar setup (except for scipy which is 0.12.0 here) and I cannot reproduce the failure.

Contributor

chyikwei commented Sep 22, 2013

I think the issue I met is about "arpack" module.

When I set eigen_solver=None or eigen_solver="arpack" , output from spectral_embedding function looks weird:

[[ -2.32766652e+08 -1.39966513e+00]
[ 3.86210625e+08 -1.54720798e+00]
[ -1.48187517e+08 -9.47395362e-01]
[ -5.64702377e+07 -7.93513447e-01]
[ 1.21781087e+08 -1.27888857e-01]
[ 1.59543207e+08 8.04107066e-01]
[ -2.11367269e+08 9.66087598e-01]
[ -1.47024602e+08 9.62468083e-01]
[ -1.85680793e+06 9.62468083e-01]
[ 1.30138155e+08 1.12053995e+00]]

But when I set eigen_solver="amg" , the output of spectral_embedding looks more reasonable:

[[-3.0003491 -0.76365248] [-3.31662479 -0.91420026] [-2.03085492 -0.18276992] [-1.70099068 0.57040841] [-0.27414501 3.16227766] [ 1.72369937 -0.85514476] [ 2.07092395 1.20605346] [ 2.06316509 -0.76410269] [ 2.06316509 -0.76410269] [ 2.40201099 -0.69476673]]

And the output can be classified correctly by kmeans and pass the test.
Any suggestion how to fix it?

Owner

ogrisel commented Sep 25, 2013

Indeed this is a known bug of OSX that will be fixed in 10.9: See #2431 and scipy/scipy#2547. What is weird is that I don't reproduce the failure myself...

Owner

ogrisel commented Sep 25, 2013

Actually I was mistaken: it has also been fixed in the scipy wrapper itself in the master branch and schedule for release 0.13.

Contributor

chyikwei commented Sep 26, 2013

The issue is fixed after I installed scipy '0.14.0.dev-1164439'
But the test still failed at test_connect_regions_with_grid (same as issue #1968)

Owner

ogrisel commented Sep 26, 2013

I could reproduce this issue (with test test_spectral_clustering_sparse) on osx with scipy 0.12.0 but apparently only with python 3, not python 2.7... weird.

Owner

ogrisel commented Sep 26, 2013

I cannot reproduce the test_connect_regions_with_grid failure myself...

Owner

ogrisel commented Sep 26, 2013

Ah ok test_connect_regions_with_grid is only there in scipy master, not in 0.12.0. If you have time to investigate this one your input on #1968 would surely be appreciated.

Contributor

dengemann commented Oct 17, 2013

@ogrisel FYI I can reproduce this with recent Anaconda on Mac OS 1.8 and numpy 1.8 and sicpy 0.12.
Any news on this?

Contributor

dengemann commented Oct 17, 2013

ahhh sorry numpy 1.7.1

Contributor

chyikwei commented Oct 17, 2013

@dengemann The test fail is fixed after I re-install scipy 0.12 from its source code.

What I did is

  1. remove scipy
  2. git clone scipy repository
  3. checkout to tag 0.12.0rc1
  4. install

Maybe you can give it a try.

@amueller amueller added this to the 0.15.1 milestone Jul 18, 2014

Owner

ogrisel commented Jul 29, 2014

I cannot reproduce this failure when building scipy against the Accelerate of OSX 10.9.

Closing.

@ogrisel ogrisel closed this Jul 29, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment