Unsorted eigenvalues returned by linalg.eigh()?

[xiumingzhang]

Eigenvalues returned by linalg.eigh() are supposed to be "in ascending order", but it returned me an unsorted array of eigenvalues for my big 2D array (of shape (88329, 88329)).

(Pdb) pp eig_vals
array([  6.93335480e-310,   6.93335480e-310,   1.76908623e-316, ...,
         1.25299807e+001,   1.34932302e+001,   3.49126548e-318])

Also, I'm worried about super large exponents, like e-310, although they will not matter if I'm using only a few top eigenvalues.

I guess this is due to the fact that I have way more dimensions (88329) than data points (517)?

If this is so, maybe a warning should be issued cautioning the user?

[charris]

Need more information, in particular, what function did you call to get the eigenvalues and what platform are you using.

[xiumingzhang]

@charris

Function called: numpy.linalg.eigh().

I'm on Ubuntu 14.04.5 LTS, using

>>> np.__version__
'1.12.1'

[charris]

Hmm, the small numbers are denormals, I wonder if that matters. It would be good to have a simple way to reproduce this. It is unusual to see such numbers unless viewing uninitialized memory.

[pv]

Iirc the sort is done by lapack, so it may be specific to the one used. 31.8.2017 22.28 "Charles Harris" <notifications@github.com> kirjoitti: Hmm, the small numbers are denormals, I wonder if that matters. It would be good to have a simple way to reproduce this. It is unusual to see such numbers unless viewing uninitialized memory. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9634 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACI5kf63Rcxj4dnaemUvZ-g4EEvUpGrks5sdwlggaJpZM4PIO-x> .

[charris]

@pv could be, but the ascending order is part of the reference implementation documentation of DSYEVD. @xiumingzhang Can you tell what you are using for LAPACK routines? Try np.__config__.show().

[charris]

@xiumingzhang An array with shape (88329, 517) isn't square, so should fail with error. What are you really doing?

[xiumingzhang]

@charris

>>> np.__config__.show()
blas_mkl_info:
    libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
    library_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/include']
blas_opt_info:
    libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
    library_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/include']
lapack_mkl_info:
    libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
    library_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/include']
lapack_opt_info:
    libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
    library_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/software/Anaconda3-4.3.1-Linux-x86_64/include']

Sorry, that was a typo. The array size is (88329, 88329).

Thanks for looking into the problem!