Stack smashing detected when running linalg.eig

[djmarti]

Hi everyone,

I get a "stack smashing detected" error and a subsequent crash when I
call linalg.eig with a sufficiently large array (N=5000). I didn't have
this problem in previous versions. This may well be a liblapack problem
rather than a scipy problem, but I thought it would be worth sharing
anyway.

I am using scipy 0.16.1, numpy 1.9.2, and lapack 3.5.0-5, running on
Linux 4.2.0 (Debian 4.2.6-1 (2015-11-10) x86_64 GNU/Linux). This is
backtrace I obtain with a minimal example:

(gdb) run -c "import numpy as np; import scipy.linalg as LA; a = np.random.randn(5000,5000); l, R = LA.eig(a)"
Starting program: /usr/bin/python -c "import numpy as np; import scipy.linalg as LA; a = np.random.randn(5000,5000); l, R = LA.eig(a)"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff3a17700 (LWP 14095)]
...
[New Thread 0x7fffbea01700 (LWP 14117)]
*** stack smashing detected ***: /usr/bin/python terminated

#0  0x00007ffff6f28107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff6f294e8 in __GI_abort () at abort.c:89
#2  0x00007ffff6f66214 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff7056a4b "*** %s ***: %s terminated\n") at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff6fe94e7 in __GI___fortify_fail (msg=msg@entry=0x7ffff7056a33 "stack smashing detected") at fortify_fail.c:31
#4  0x00007ffff6fe94b0 in __stack_chk_fail () at stack_chk_fail.c:28
#5  0x00007ffff62b93be in dgemv_ () from /usr/lib/libblas.so.3
#6  0x00007fffb8b2f7b4 in dlahr2_ () from /usr/lib/liblapack.so.3
#7  0x00007fffb8ad95bd in dgehrd_ () from /usr/lib/liblapack.so.3
#8  0x00007fffb8b45262 in dlaqr3_ () from /usr/lib/liblapack.so.3
#9  0x00007fffb8b41248 in dlaqr0_ () from /usr/lib/liblapack.so.3
#10 0x00007fffb8b1c929 in dhseqr_ () from /usr/lib/liblapack.so.3
#11 0x00007fffb8ad3d05 in dgeev_ () from /usr/lib/liblapack.so.3
#12 0x00007fffb76343db in ?? () from /usr/lib/python2.7/dist-packages/scipy/linalg/_flapack.x86_64-linux-gnu.so
#13 0x00000000004bff6f in PyEval_EvalFrameEx ()
#14 0x00000000004b8556 in PyEval_EvalCodeEx ()
#15 0x00000000004c009a in PyEval_EvalFrameEx ()
#16 0x00000000004b8556 in PyEval_EvalCodeEx ()
#17 0x0000000000519198 in PyRun_StringFlags ()
#18 0x0000000000519a5a in PyRun_SimpleStringFlags ()
#19 0x0000000000491a7d in Py_Main ()
#20 0x00007ffff6f14b45 in __libc_start_main (main=0x491670 <main>, argc=3, argv=0x7fffffffdbd8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7fffffffdbc8) at libc-start.c:287
#21 0x000000000049159b in _start ()

Best,

dani

[pv]

Fairly surely an issue with your blas/lapack libraries. You can try e.g. to swapping to a different blas library and checking the input parameters to dgeev in a debugger.

[juliantaylor]

debian switched to -fstack-protector-strong so its more aggressive than before
which blas is being used? looks like netlib blas to me

update-alternatives --display libblas.so.3

[djmarti]

Thanks for your answers. I was using openblas and now I switched to blas. It seems to work fine now! Here is my current choice of the library

$ update-alternatives --display libblas.so.3
libblas.so.3 - manual mode
  link best version is /usr/lib/openblas-base/libblas.so.3
  link currently points to /usr/lib/libblas/libblas.so.3
  link libblas.so.3 is /usr/lib/libblas.so.3
  slave libblas.so.3gf is /usr/lib/libblas.so.3gf
/usr/lib/libblas/libblas.so.3 - priority 10
/usr/lib/openblas-base/libblas.so.3 - priority 40
  slave libblas.so.3gf: /usr/lib/openblas-base/libblas.so.3

Again, thanks for your quick help.

[juliantaylor]

hm I can't reproduce it
which version of debian are you using?
what is your cpu and number of cpus?

[djmarti]

I am using Debian testing (stretch) on a system with 24 cores Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz

$ cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 45
model name  : Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
stepping    : 7
microcode   : 0x70e
cpu MHz     : 1200.000
cache size  : 15360 KB
physical id : 0
siblings    : 12
core id     : 0
cpu cores   : 6
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid xsaveopt
bugs        :
bogomips    : 3990.30
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:
...