Segfaults in matrix_modn_sparse in Sage built with clang #24716

simon-king-jena · 2018-02-12T14:17:34Z

When building Sage on Ubuntu with clang (see #24701, #24705 etc), a bunch of doctests fails (see comments below). They have a common cause in matrix_modn_sparse.

I guess it should be fixed! And I hope "porting" is the correct component for it.

CC: @rwst @kiwifb @jdemeyer

Component: porting

Reviewer: François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/24716

simon-king-jena · 2018-02-12T14:24:49Z

comment:1

Attachment: test.log

Looking at the attachment: test.log, it seems that all errors are caused by a segfault in matrix_modn_sparse (the one timeout error is an artefact of me closing my laptop and bringing it to my office).

So, in fact only matrix_modn_sparse needs to be fixed.

simon-king-jena · 2018-02-12T15:32:33Z

comment:2

sage: a = matrix(Integers(37),3,3,range(9),sparse=True)
sage: a._rank_linbox(0)
---------------------------------------------------------------------------
SignalError                               Traceback (most recent call last)
<ipython-input-12-9dcccafd29df> in <module>()
----> 1 a._rank_linbox(Integer(0))

/home/king/Sage/clang/sage/local/lib/python2.7/site-packages/sage/misc/decorators.pyc in wrapper(*args, **kwds)
    720                     kwds[new_name] = kwds[old_name]
    721                     del kwds[old_name]
--> 722             return func(*args, **kwds)
    723 
    724         return wrapper

/home/king/Sage/clang/sage/src/sage/matrix/matrix_modn_sparse.pyx in sage.matrix.matrix_modn_sparse.Matrix_modn_sparse._rank_linbox (build/cythonized/sage/matrix/matrix_modn_sparse.c:9008)()
    728                 return x
    729             self._init_linbox()
--> 730             sig_on()
    731             # the returend pivots list is currently wrong
    732             #r, pivots = linbox.rank(1)

SignalError: Segmentation fault
sage: a._rank_linbox(1)
---------------------------------------------------------------------------
SignalError                               Traceback (most recent call last)
<ipython-input-13-3359e7e09a77> in <module>()
----> 1 a._rank_linbox(Integer(1))

/home/king/Sage/clang/sage/local/lib/python2.7/site-packages/sage/misc/decorators.pyc in wrapper(*args, **kwds)
    720                     kwds[new_name] = kwds[old_name]
    721                     del kwds[old_name]
--> 722             return func(*args, **kwds)
    723 
    724         return wrapper

/home/king/Sage/clang/sage/src/sage/matrix/matrix_modn_sparse.pyx in sage.matrix.matrix_modn_sparse.Matrix_modn_sparse._rank_linbox (build/cythonized/sage/matrix/matrix_modn_sparse.c:9008)()
    728                 return x
    729             self._init_linbox()
--> 730             sig_on()
    731             # the returend pivots list is currently wrong
    732             #r, pivots = linbox.rank(1)

SignalError: Segmentation fault

So, perhaps something is wrong in linbox when built with clang.

kiwifb · 2018-02-12T20:02:34Z

comment:3

Just to be clear, which version of ubuntu and which version of clang? I have successfully doctested a sage build on Gentoo with clang-5.0.1 and older versions of sage with clang-4 and 3.9 but some stuff has changed since then.

simon-king-jena · 2018-02-12T20:53:56Z

comment:4

Replying to @kiwifb:

Just to be clear, which version of ubuntu and which version of clang? I have successfully doctested a sage build on Gentoo with clang-5.0.1 and older versions of sage with clang-4 and 3.9 but some stuff has changed since then.

king@king-C70-B:~/Sage/clang/sage$ clang --version
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
king@king-C70-B:~/Sage/clang/sage$ cat /etc/issue
Ubuntu 16.04.3 LTS \n \l

simon-king-jena · 2018-02-12T21:38:01Z

comment:5

To be on the safe side, I am rebuilding Sage now, as yesterday I had interrupted the build process at some point and changed one environment variable.

kiwifb · 2018-02-12T21:41:23Z

comment:6

Replying to @simon-king-jena:

To be on the safe side, I am rebuilding Sage now, as yesterday I had interrupted the build process at some point and changed one environment variable.

Thanks for doing this. It could really be something with clang-3.8, we haven't really tested that version in some time. The ticket to enable clang had a long life (and it didn't quite make it to 500 comments).

simon-king-jena · 2018-02-13T06:02:55Z

comment:7

Rebuilding Sage with the branch from #24696, with clang-3.8, and with

export CC="clang"
export CXX="clang++"
export CLANG_DEFAULT_CXX_STDLIB="libc++"

the error persists. So, something is going wrong.

kiwifb · 2018-02-13T06:22:39Z

comment:8

Right, can you attach the build log of linbox and if you can generate it, its config.log.

rwst · 2018-02-13T06:47:57Z

comment:9

Also helpful would be to start sage -gdb, make it crash, input bt on the prompt, and post the resulting backtrace (the first page or so, until python functions show).

simon-king-jena · 2018-02-13T07:00:18Z

Attachment: linbox-1.5.2.log

simon-king-jena · 2018-02-13T07:00:53Z

comment:10

Replying to @kiwifb:

Right, can you attach the build log of linbox

Done.

and if you can generate it, its config.log.

How to generate it?

simon-king-jena · 2018-02-13T07:06:20Z

comment:11

Replying to @rwst:

Also helpful would be to start sage -gdb, make it crash, input bt on the prompt, and post the resulting backtrace (the first page or so, until python functions show).

When I start ./sage -gdb, I see No module named Cython during startup:

$ ./sage -gdb
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.2.beta5, Release Date: 2018-02-09               │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...done.
Traceback (most recent call last):
  File "<string>", line 11, in <module>
ImportError: No module named 'Cython'
/tmp/tmp4PZPMk:19: Error in sourced command file:
Error while executing Python code.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff0427700 (LWP 7526)]
sage: [New Thread 0x7ffea328c700 (LWP 7534)]
[Thread 0x7ffea328c700 (LWP 7534) exited]
[New Thread 0x7ffea328c700 (LWP 7535)]
[Thread 0x7ffea328c700 (LWP 7535) exited]
[New Thread 0x7ffea328c700 (LWP 7536)]
[Thread 0x7ffea328c700 (LWP 7536) exited]
[New Thread 0x7ffea328c700 (LWP 7537)]

Quite odd, from my point of view. So, why does Sage even start if it cannot find Cython?

Note, however, that I see the same ImportError when starting sage -gdb when using gcc.

Then, after generating the segfault, I get:

sage: a._rank_linbox(1)

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007fffe11e95d9 in std::ostream::sentry::sentry(std::ostream&) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007fffe11e95d9 in std::ostream::sentry::sentry(std::ostream&) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007fffe11e9d59 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x00007ffec2828425 in std::operator<< <std::char_traits<char> > (__out=..., 
    __s=<optimized out>)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/ostream:561
#3  LinBox::GaussDomain<Givaro::Modular<unsigned int, unsigned int> >::NoReordering<LinBox::SparseMatrix<Givaro::Modular<unsigned int, unsigned int>, LinBox::SparseMatrixFormat::SparseSeq> > (
    this=0x7fffffffb800, res=@0x7fffffffb810: 140737351561824, determinant=<optimized out>, 
    LigneA=..., Ni=3, Nj=3) at ../../linbox/algorithms/gauss/gauss.inl:797
#4  0x00007ffec28269a2 in linbox_modn_sparse_matrix_rank (modulus=<optimized out>, numrows=3, 
    numcols=3, rows=<optimized out>, gauss=1) at linbox-sage.C:122
#5  0x00007ffec0ab34ea in __pyx_f_4sage_4libs_6linbox_6linbox_18Linbox_modn_sparse_rank (
    __pyx_v_self=<optimized out>, __pyx_v_gauss=-134445312)
    at build/cythonized/sage/libs/linbox/linbox.cpp:2826
#6  0x00007ffec482021f in __pyx_pf_4sage_6matrix_18matrix_modn_sparse_18Matrix_modn_sparse_28_rank_linbox (__pyx_v_self=0x7ffea2a83af0, __pyx_v_algorithm=0x7ffea34faea0)
    at build/cythonized/sage/matrix/matrix_modn_sparse.c:9018
#7  0x00007ffff7a3e601 in PyObject_Call (func=0x7ffea3791e60, arg=0x7ffea3769090, 
    kw=0x7ffea2211a28) at Objects/abstract.c:2547
#8  0x00007ffff7ae50c2 in PyEval_CallObjectWithKeywords (func=0x7ffea3791e60, 
    arg=0x7ffea3769090, kw=0x7ffea2211a28) at Python/ceval.c:4226
#9  0x00007ffff7a55161 in methoddescr_call (descr=<optimized out>, args=0x7ffea3769090, 
    kwds=0x7ffea2211a28) at Objects/descrobject.c:247
#10 0x00007ffff7a3e601 in PyObject_Call (func=0x7ffec4eb32d8, arg=0x7ffea2213320, 
    kw=0x7ffea2211a28) at Objects/abstract.c:2547
#11 0x00007ffff7ae129b in ext_do_call (func=<optimized out>, nk=<optimized out>, 
    pp_stack=<optimized out>, flags=<optimized out>, na=<optimized out>) at Python/ceval.c:4671
#12 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3033
#13 0x00007ffff7ad8914 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=<optimized out>, defs=<optimized out>, defcount=<optimized out>, 
    closure=<optimized out>) at Python/ceval.c:3589
#14 0x00007ffff7ae5672 in fast_function (func=<optimized out>, pp_stack=0x7fffffffbca8, 
    n=<optimized out>, na=0, nk=<optimized out>) at Python/ceval.c:4452
#15 0x00007ffff7ae023c in call_function (oparg=<optimized out>, pp_stack=<optimized out>)
    at Python/ceval.c:4377
#16 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2994
#17 0x00007ffff7ad8914 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=<optimized out>, defs=<optimized out>, defcount=<optimized out>, 
    closure=<optimized out>) at Python/ceval.c:3589
#18 0x00007ffff7ad8f26 in PyEval_EvalCode (co=0x7ffea354fd30, globals=0x7ffff04426e0, 
    locals=0x7ffff04426e0) at Python/ceval.c:669
#19 exec_statement (f=<optimized out>, prog=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>) at Python/ceval.c:5075
#20 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2110
#21 0x00007ffff7ad8914 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=<optimized out>, defs=<optimized out>, defcount=<optimized out>, 
    closure=<optimized out>) at Python/ceval.c:3589
#22 0x00007ffff7ae5672 in fast_function (func=<optimized out>, pp_stack=0x7fffffffc0d8, 
    n=<optimized out>, na=0, nk=<optimized out>) at Python/ceval.c:4452
#23 0x00007ffff7ae023c in call_function (oparg=<optimized out>, pp_stack=<optimized out>)
    at Python/ceval.c:4377
#24 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2994
#25 0x00007ffff7ad8914 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, 
    locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=<optimized out>, defs=<optimized out>, defcount=<optimized out>, 
    closure=<optimized out>) at Python/ceval.c:3589
#26 0x00007ffff7ae5672 in fast_function (func=<optimized out>, pp_stack=0x7fffffffc318, 
    n=<optimized out>, na=0, nk=<optimized out>) at Python/ceval.c:4452
#27 0x00007ffff7ae023c in call_function (oparg=<optimized out>, pp_stack=<optimized out>)
    at Python/ceval.c:4377

rwst · 2018-02-13T07:10:25Z

comment:12

Well linbox/algorithms/gauss/gauss.inl (INL? what's this?) seems to have been linked with libstdc++.

kiwifb · 2018-02-13T07:43:08Z

comment:13

Replying to @rwst:

Well linbox/algorithms/gauss/gauss.inl (INL? what's this?) seems to have been linked with libstdc++.

.inl is for inline. The presence of libstdc++ would explains things. What is ldd -r /home/king/Sage/clang/sage/local/lib/python2.7/site-packages/sage/matrix/matrix_modn_sparse.so and ldd -r /home/king/Sage/clang/sage/local/lib/liblinbox.so and possibly ldd -r /home/king/Sage/clang/sage/local/lib/liblinbox_sage.so.

simon-king-jena · 2018-02-13T07:43:43Z

comment:14

Replying to @rwst:

Well linbox/algorithms/gauss/gauss.inl (INL? what's this?) seems to have been linked with libstdc++.

I unpacked the linbox upstream tarball, and then

$ grep "stdc++" -R .
./linbox-1.5.2/configure:					MAPLE_LIBS="-L${MAPLE_BIN} -lmaplec -lstdc++"
./linbox-1.5.2/macros/maple-check.m4:					MAPLE_LIBS="-L${MAPLE_BIN} -lmaplec -lstdc++"

So, is -lstdc++ hardcoded in linbox?

rwst · 2018-02-13T07:47:26Z

comment:15

Replying to @simon-king-jena:

So, is -lstdc++ hardcoded in linbox?

Apparently, because my ldd -r liblinbox.so is linked with it too. But I don't have these crashes, must be luck.

simon-king-jena · 2018-02-13T07:47:43Z

comment:16

Replying to @kiwifb:

.inl is for inline. The presence of libstdc++ would explains things. What is ldd -r /home/king/Sage/clang/sage/local/lib/python2.7/site-packages/sage/matrix/matrix_modn_sparse.so and ldd -r /home/king/Sage/clang/sage/local/lib/liblinbox.so and possibly ldd -r /home/king/Sage/clang/sage/local/lib/liblinbox_sage.so.

$ ldd -r /home/king/Sage/clang/sage/local/lib/python2.7/site-packages/sage/matrix/matrix_modn_sparse.so
	linux-vdso.so.1 =>  (0x00007ffd1dbfb000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007ffb45c92000)
	libpython2.7.so.1.0 => /home/king/Sage/clang/sage/local/lib/libpython2.7.so.1.0 (0x00007ffb458ac000)
	libpari-gmp-2.10.so.0 => /home/king/Sage/clang/sage/local/lib/libpari-gmp-2.10.so.0 (0x00007ffb44d3c000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ffb44b1f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffb44755000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ffb44551000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007ffb4434e000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffb44045000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ffb46149000)

and

$ ldd -r /home/king/Sage/clang/sage/local/lib/liblinbox.so
	linux-vdso.so.1 =>  (0x00007ffcc7af9000)
	libntl.so.33 => /home/king/Sage/clang/sage/local/lib/libntl.so.33 (0x00007f061fd3e000)
	libflint.so.13 => /home/king/Sage/clang/sage/local/lib/libflint.so.13 (0x00007f061f7bc000)
	libfflas.so.1 => /home/king/Sage/clang/sage/local/lib/libfflas.so.1 (0x00007f061f4a8000)
	libffpack.so.1 => /home/king/Sage/clang/sage/local/lib/libffpack.so.1 (0x00007f061f160000)
	libopenblas.so.0 => /home/king/Sage/clang/sage/local/lib/libopenblas.so.0 (0x00007f061e37d000)
	libgivaro.so.9 => /home/king/Sage/clang/sage/local/lib/libgivaro.so.9 (0x00007f061e126000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007f061de9a000)
	libgmpxx.so.8 => /home/king/Sage/clang/sage/local/lib/libgmpxx.so.8 (0x00007f061dc95000)
	libmpfr.so.6 => /home/king/Sage/clang/sage/local/lib/libmpfr.so.6 (0x00007f061da22000)
	libiml.so.0 => /home/king/Sage/clang/sage/local/lib/libiml.so.0 (0x00007f061d7fb000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f061d479000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f061d170000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f061cda6000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f061cb90000)
	libgf2x.so.1 => /home/king/Sage/clang/sage/local/lib/libgf2x.so.1 (0x00007f061c96d000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f061c750000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f062040f000)
	libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f061c425000)
	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f061c1e6000)

and (note that it is liblinboxsage.so, not liblinbox_sage.so)

$ ldd -r /home/king/Sage/clang/sage/local/lib/liblinboxsage.so
	linux-vdso.so.1 =>  (0x00007ffeacea5000)
	libntl.so.33 => /home/king/Sage/clang/sage/local/lib/libntl.so.33 (0x00007f55cff1d000)
	libmpfr.so.6 => /home/king/Sage/clang/sage/local/lib/libmpfr.so.6 (0x00007f55cfcaa000)
	libiml.so.0 => /home/king/Sage/clang/sage/local/lib/libiml.so.0 (0x00007f55cfa83000)
	libflint.so.13 => /home/king/Sage/clang/sage/local/lib/libflint.so.13 (0x00007f55cf501000)
	libfflas.so.1 => /home/king/Sage/clang/sage/local/lib/libfflas.so.1 (0x00007f55cf1ed000)
	libffpack.so.1 => /home/king/Sage/clang/sage/local/lib/libffpack.so.1 (0x00007f55ceea5000)
	libopenblas.so.0 => /home/king/Sage/clang/sage/local/lib/libopenblas.so.0 (0x00007f55ce0c2000)
	libgivaro.so.9 => /home/king/Sage/clang/sage/local/lib/libgivaro.so.9 (0x00007f55cde6b000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007f55cdbdf000)
	libgmpxx.so.8 => /home/king/Sage/clang/sage/local/lib/libgmpxx.so.8 (0x00007f55cd9da000)
	liblinbox.so.0 => /home/king/Sage/clang/sage/local/lib/liblinbox.so.0 (0x00007f55cd7cc000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f55cd44a000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f55cd141000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f55ccd77000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f55ccb61000)
	libgf2x.so.1 => /home/king/Sage/clang/sage/local/lib/libgf2x.so.1 (0x00007f55cc93e000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f55cc721000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f55d07d1000)
	libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f55cc3f6000)
	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f55cc1b7000)

kiwifb · 2018-02-13T08:11:26Z

comment:17

Ok ldd -r for libgmpxx.so, libntl.so and libgivaro.so.

simon-king-jena · 2018-02-13T08:26:03Z

comment:18

Replying to @kiwifb:

Ok ldd -r for libgmpxx.so, libntl.so and libgivaro.so.

$ ldd -r /home/king/Sage/clang/sage/local/lib/libgmpxx.so
	linux-vdso.so.1 =>  (0x00007ffde31fe000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007ff46798f000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff46760d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff467304000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff466f3a000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff466d24000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff467e20000)

and

$ ldd -r /home/king/Sage/clang/sage/local/lib/libntl.so
	linux-vdso.so.1 =>  (0x00007fffedff9000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007ffb5a040000)
	libgf2x.so.1 => /home/king/Sage/clang/sage/local/lib/libgf2x.so.1 (0x00007ffb59e1d000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ffb59a9b000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffb59792000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffb593c8000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ffb591b2000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ffb5a78f000)

and

$ ldd -r /home/king/Sage/clang/sage/local/lib/libgivaro.so
	linux-vdso.so.1 =>  (0x00007ffdca500000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007f0bd6457000)
	libgmpxx.so.8 => /home/king/Sage/clang/sage/local/lib/libgmpxx.so.8 (0x00007f0bd6252000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0bd5ed0000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0bd5bc7000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0bd57fd000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0bd55e7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f0bd693a000)

and by the way also

$ ldd -r /home/king/Sage/clang/sage/local/lib/libsingular_resources.so 
	linux-vdso.so.1 =>  (0x00007fff1fda9000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f49c38f1000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f49c35e8000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f49c321e000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f49c3008000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f49c3e79000)

So, there is libstdc++ everywhere :-(.

kiwifb · 2018-02-13T08:27:33Z

comment:19

Is there libc++ anywhere?

kiwifb · 2018-02-13T08:28:38Z

comment:20

ldd -r *.so | grep "libc++.so" should find if there is any.

simon-king-jena · 2018-02-13T08:30:07Z

comment:21

Replying to @kiwifb:

Is there libc++ anywhere?

No.

$ ldd -r /home/king/Sage/clang/sage/local/lib/*.so | grep "libc++.so"

shows nothing. Hm.

simon-king-jena · 2018-02-13T08:33:52Z

comment:22

Replying to @simon-king-jena:

Rebuilding Sage with the branch from #24696, with clang-3.8, and with
export CC="clang"
export CXX="clang++"
export CLANG_DEFAULT_CXX_STDLIB="libc++"

Is it perhaps the case that one needs additional flags?

kiwifb · 2018-02-13T08:38:12Z

comment:23

I am wondering if CLANG_DEFAULT_CXX_STDLIB has any effect on your clang install. Let's see the log of ntl with the following LDFLAGS=-Wl,--verbose ./sage -f ntl and delete the previous log so it is a clean one.

rwst · 2018-02-13T08:41:51Z

comment:24

Replying to @simon-king-jena:

Replying to @simon-king-jena:
Rebuilding Sage with the branch from #24696, with clang-3.8, and with
export CC="clang"
export CXX="clang++"
export CLANG_DEFAULT_CXX_STDLIB="libc++"
Is it perhaps the case that one needs additional flags?

I should really have checked the so's. The additional flag I think would be LDFLAGS="$LDFLAGS -lc++ --stdlib=libc++" because when I did that I got libc++ in the so's.

rwst · 2018-02-13T08:43:34Z

comment:25

Replying to @rwst:

I should really have checked the so's. The additional flag I think would be LDFLAGS="$LDFLAGS -lc++ --stdlib=libc++" because when I did that I got libc++ in the so's.

But then gfortran will complain in scipy, see #24710. Maybe without stdlib?

rwst · 2018-02-13T08:57:43Z

comment:26

I just built mpir and gf2x then ntl with `LDFLAGS="$LDFLAGS -lc++ --stlib=ĺibc++" and libc++ is in libntl but libstdc++ too. libmpir.so does not have libstdc++, so where does it come from?

simon-king-jena · 2018-02-13T08:58:22Z

comment:27

Replying to @rwst:

Replying to @rwst:

I should really have checked the so's. The additional flag I think would be LDFLAGS="$LDFLAGS -lc++ --stdlib=libc++" because when I did that I got libc++ in the so's.

But then gfortran will complain in scipy, see #24710. Maybe without stdlib?

I have enough time to rebuild from scratch, and (as I did before) with #24696 merged with #24710.

So, perhaps it was too early to close #24710 as invalid.

kiwifb · 2018-02-13T08:58:44Z

comment:28

libmpir is pure C, try libmpirxx

simon-king-jena · 2018-02-13T08:59:14Z

comment:29

Replying to @rwst:

I just built mpir and gf2x then ntl with `LDFLAGS="$LDFLAGS -lc++ --stlib=ĺibc++" and libc++ is in libntl but libstdc++ too. libmpir.so does not have libstdc++, so where does it come from?

See comment:14, ntl seems to hard-code the use of libstdc++

rwst · 2018-02-13T09:00:10Z

comment:30

Replying to @kiwifb:

libmpir is pure C, try libmpirxx

Ah there it is, too, so mpir is it.

simon-king-jena · 2018-02-13T09:09:02Z

Attachment: ntl-10.3.0.log

simon-king-jena · 2018-02-13T09:11:01Z

comment:31

The log obtained with

LDFLAGS=-Wl,--verbose ./sage -f ntl

is attached, and

$ ldd -r /home/king/Sage/clang/sage/local/lib/libntl.so
	linux-vdso.so.1 =>  (0x00007fff471f1000)
	libgmp.so.23 => /home/king/Sage/clang/sage/local/lib/libgmp.so.23 (0x00007f86ea47a000)
	libgf2x.so.1 => /home/king/Sage/clang/sage/local/lib/libgf2x.so.1 (0x00007f86ea257000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f86e9ed5000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f86e9bcc000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f86e9802000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f86e95ec000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f86eabc9000)

So, I guess one needs to fix the ntl upstream tarball.

kiwifb · 2018-02-13T09:16:38Z

comment:32

But I have zero problem with a clang defaulting to libc++, even so according to this -lstdc++ is passed at linking time. I have to rebuild things again and inspect the libraries. By the way according to the log you posted the linker never even looked for libc++.so.

rwst · 2018-02-13T09:19:39Z

comment:33

Replying to @kiwifb:

But I have zero problem with a clang defaulting to libc++, even so according to this -lstdc++ is passed at linking time.

But is it after that in the so? As said I have it in all so's and don't get crashes either. Remember also libc++ is used when compiling as well not only linking.

kiwifb · 2018-02-13T10:04:17Z

comment:34

That's what it looks like here

fbissey@moonloop ~/sandbox/git-fork/sage-clang5 $ ldd -r local/lib/libmpirxx.so
	linux-vdso.so.1 (0x00007ffed9b10000)
	libmpir.so.23 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libmpir.so.23 (0x00007f4cd733d000)
	libc++.so.1 => /usr/lib64/libc++.so.1 (0x00007f4cd7083000)
	libc++abi.so.1 => /usr/lib64/libc++abi.so.1 (0x00007f4cd6e59000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libgcc_s.so.1 (0x00007f4cd6c42000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f4cd693b000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f4cd6588000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4cd6369000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4cd77cc000)
fbissey@moonloop ~/sandbox/git-fork/sage-clang5 $ ldd -r local/lib/libntl.so
	linux-vdso.so.1 (0x00007fff9c1fd000)
	libgmp.so.23 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libgmp.so.23 (0x00007f6f93607000)
	libgf2x.so.1 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libgf2x.so.1 (0x00007f6f933e4000)
	libc++.so.1 => /usr/lib64/libc++.so.1 (0x00007f6f9312a000)
	libc++abi.so.1 => /usr/lib64/libc++abi.so.1 (0x00007f6f92f00000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libgcc_s.so.1 (0x00007f6f92ce9000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f6f929e2000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f6f9262f000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6f92410000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6f93d7c000)
fbissey@moonloop ~/sandbox/git-fork/sage-clang5 $ ldd -r local/lib/liblinbox.so
	linux-vdso.so.1 (0x00007ffe1dbfd000)
	libntl.so.33 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libntl.so.33 (0x00007f24fc490000)
	libflint.so.13 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libflint.so.13 (0x00007f24fbf10000)
	libfflas.so.1 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libfflas.so.1 (0x00007f24fbbf4000)
	libffpack.so.1 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libffpack.so.1 (0x00007f24fb8a4000)
	libopenblas.so.0 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libopenblas.so.0 (0x00007f24faac5000)
	libgivaro.so.9 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libgivaro.so.9 (0x00007f24fa86b000)
	libgmp.so.23 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libgmp.so.23 (0x00007f24fa5e0000)
	libgmpxx.so.8 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libgmpxx.so.8 (0x00007f24fa3dc000)
	libmpfr.so.6 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libmpfr.so.6 (0x00007f24fa165000)
	libiml.so.0 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libiml.so.0 (0x00007f24f9f3f000)
	libc++.so.1 => /usr/lib64/libc++.so.1 (0x00007f24f9c85000)
	libc++abi.so.1 => /usr/lib64/libc++abi.so.1 (0x00007f24f9a5b000)
	libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libgcc_s.so.1 (0x00007f24f9844000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f24f953d000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f24f918a000)
	libgf2x.so.1 => /home/fbissey/sandbox/git-fork/sage-clang5/local/lib/libgf2x.so.1 (0x00007f24f8f67000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f24f8d48000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f24fcb8c000)
	libgfortran.so.4 => /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libgfortran.so.4 (0x00007f24f896c000)
	libquadmath.so.0 => /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libquadmath.so.0 (0x00007f24f872c000)

no libstdc++ anywhere.

kiwifb · 2018-02-13T10:08:11Z

comment:35

And as you can see, I don't have -lstdc++ added when I link nlt, but i have -lc++

libtool: link: clang++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/crtbeginS.o  .libs/FFT.o .libs/FacVec.o .libs/GF2.o .libs/GF2E.o .libs/GF2EX.o .libs/GF2EXFactoring.o .libs/GF2X.o .libs/GF2X1.o .libs/GF2XFactoring.o .libs/GF2XVec.o .libs/GetTime.o .libs/GetPID.o .libs/HNF.o .libs/ctools.o .libs/LLL.o .libs/LLL_FP.o .libs/LLL_QP.o .libs/LLL_RR.o .libs/LLL_XD.o .libs/RR.o .libs/WordVector.o .libs/ZZ.o .libs/ZZVec.o .libs/ZZX.o .libs/ZZX1.o .libs/ZZXCharPoly.o .libs/ZZXFactoring.o .libs/ZZ_p.o .libs/ZZ_pE.o .libs/ZZ_pEX.o .libs/ZZ_pEXFactoring.o .libs/ZZ_pX.o .libs/ZZ_pX1.o .libs/ZZ_pXCharPoly.o .libs/ZZ_pXFactoring.o .libs/fileio.o .libs/lip.o .libs/lzz_p.o .libs/lzz_pE.o .libs/lzz_pEX.o .libs/lzz_pEXFactoring.o .libs/lzz_pX.o .libs/lzz_pX1.o .libs/lzz_pXCharPoly.o .libs/lzz_pXFactoring.o .libs/mat_GF2.o .libs/mat_GF2E.o .libs/mat_RR.o .libs/mat_ZZ.o .libs/mat_ZZ_p.o .libs/mat_ZZ_pE.o .libs/mat_lzz_p.o .libs/mat_lzz_pE.o .libs/mat_poly_ZZ.o .libs/mat_poly_ZZ_p.o .libs/mat_poly_lzz_p.o .libs/quad_float.o .libs/tools.o .libs/vec_GF2.o .libs/vec_GF2E.o .libs/vec_RR.o .libs/vec_ZZ.o .libs/vec_ZZ_p.o .libs/vec_ZZ_pE.o .libs/vec_lzz_p.o .libs/vec_lzz_pE.o .libs/xdouble.o .libs/G_LLL_FP.o .libs/G_LLL_QP.o .libs/G_LLL_XD.o .libs/G_LLL_RR.o .libs/thread.o .libs/BasicThreadPool.o .libs/MatPrime.o   -L/home/fbissey/sandbox/git-fork/sage-clang5/local/lib -lgmp -lgf2x -L/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0 -L/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../lib64 -L/usr/lib64/llvm/5/bin/../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../.. -L/lib -L/usr/lib -lc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../lib64/crtn.o  -march=native -O2 -Wl,-rpath -Wl,/home/fbissey/sandbox/git-fork/sage-clang5/local/lib   -Wl,-soname -Wl,libntl.so.33 -o .libs/libntl.so.33.0.0

I think libtool is figuring it out by itself.

rwst · 2018-02-13T14:46:30Z

comment:36

Maybe we should not be so absolute that clang needs libc++ to work. In #24701 I reported the exact same crash but I did not use libc++ there.

rwst · 2018-02-13T16:00:55Z

comment:37

Sage compiles here with clang-4 without options for libc++. The above tests pass without crash (full make ptestlong in the works). As you can see from #24701 on the same system with the same flags clang-3.8 produces a crashing Sage (this lead me to use libc++ from there on).

So obviously, the next attempt Simon could do would be using clang-4.

rwst · 2018-03-07T07:13:40Z

comment:38

So obviously, the next attempt Simon could do would be using clang-4.

Since that worked I propose to close this ticket.

kiwifb · 2018-03-07T07:16:54Z

comment:39

Fine with me. On linux the minimum supported version of clang needed is 4 and that's the end of it. On OS X it appears we can go back to xcode 7.

kiwifb · 2018-03-07T07:16:54Z

Reviewer: François Bissey

videlec · 2018-05-18T17:16:26Z

comment:40

closing positively reviewed duplicates

simon-king-jena added this to the sage-8.2 milestone Feb 12, 2018

simon-king-jena added c: porting labels Feb 12, 2018

This comment has been minimized.

Sign in to view

simon-king-jena changed the title ~~Make it so that Sage passes tests when built with clang~~ Segfaults in matrix_modn_sparse in Sage built with clang Feb 12, 2018

rwst added the s: needs review label Mar 7, 2018

rwst removed this from the sage-8.2 milestone Mar 7, 2018

kiwifb added s: positive review and removed s: needs review labels Mar 7, 2018

videlec removed the s: positive review label May 18, 2018

videlec closed this as completed May 18, 2018

videlec added r: wontfix and removed p: major / 3 labels May 18, 2018

Segfaults in matrix_modn_sparse in Sage built with clang #24716

Segfaults in matrix_modn_sparse in Sage built with clang #24716

Comments

simon-king-jena commented Feb 12, 2018

simon-king-jena commented Feb 12, 2018

This comment has been minimized.

simon-king-jena commented Feb 12, 2018

kiwifb commented Feb 12, 2018

simon-king-jena commented Feb 12, 2018

simon-king-jena commented Feb 12, 2018

kiwifb commented Feb 12, 2018

simon-king-jena commented Feb 13, 2018

kiwifb commented Feb 13, 2018

rwst commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

rwst commented Feb 13, 2018

kiwifb commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

rwst commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

kiwifb commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

kiwifb commented Feb 13, 2018

kiwifb commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

kiwifb commented Feb 13, 2018

rwst commented Feb 13, 2018

rwst commented Feb 13, 2018

rwst commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

kiwifb commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

rwst commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

simon-king-jena commented Feb 13, 2018

kiwifb commented Feb 13, 2018

rwst commented Feb 13, 2018

kiwifb commented Feb 13, 2018

kiwifb commented Feb 13, 2018

rwst commented Feb 13, 2018

rwst commented Feb 13, 2018

rwst commented Mar 7, 2018

kiwifb commented Mar 7, 2018

kiwifb commented Mar 7, 2018

videlec commented May 18, 2018