Kernel module not loading in latest GAP

[mtorpey]

Since a very recent change in GAP master, the Semigroups kernel module is not loaded properly:

gap> LoadPackage("semigroups", false);
#W dlopen() error: /home/mtorpey/gap/pkg/semigroups/bin/x86_64-pc-linux-gnu-gcc-default64/semigroups.so: undefined symbol: pthread_create
Error, module '/home/mtorpey/gap/pkg/semigroups/bin/x86_64-pc-linux-gnu-gcc-default64/semigroups.so' not found in
  if not LOAD_DYN( arg[1], false )  then
    Error( "no support for dynamic loading" );
fi; at /home/mtorpey/gap/lib/files.gd:579 called from 
<function "LoadDynamicModule">( <arguments> )
 called from read-eval loop at /home/mtorpey/gap/pkg/semigroups/init.g:26
Error, was not in any namespace at /home/mtorpey/gap/lib/variable.g:282 called from
LEAVE_NAMESPACE(  ); at /home/mtorpey/gap/lib/package.gi:1215 called from
ReadPackage( pkgname, "init.g" ); at /home/mtorpey/gap/lib/package.gi:1588 called from
<function "LoadPackage">( <arguments> )
 called from read-eval loop at *stdin*:1
you can 'quit;' to quit to outer loop, or
you can 'return;' to continue

This problem is introduced in GAP commit Remove the timeout code by @fingolfin. It's not clear to me how this change causes the problem, but I'm guessing we rely on some of the timeout code which has been removed.

This is causing Travis tests to fail.

[james-d-mitchell]

As far as I know we do not use any of the time out code

[james-d-mitchell]

Could this be something to do with compiling without the -pthread flag?

[fingolfin]

@james-d-mitchell Yeah, I was wondering that, too. But then I don't quite see why it worked before my PR. Also, we only use pthread_create for HPC-GAP, and as far as I can tell, your test there is not running under HPC-GAP, correct?

[james-d-mitchell]

That's right we are not using hpcgap, but we are using pthread through ourselves in the kernel module, indirectly via C++11 std::thread

[fingolfin]

Hmm. So, can you reproduce the issue locally?

[fingolfin]

For what is worth, I cannot, on Mac OS X 10.11.6

[fingolfin]

but then, there is no need for -pthread there, so I guess somebody needs to debug this on Linux.

[mtorpey]

I can reproduce the issue on my machine - in fact the output I posted is on my office PC (Debian testing).

[mtorpey]

I haven't figured out why this is broken, but it can be fixed by adding a single line to GAP's configure.ac:

diff --git a/configure.ac b/configure.ac
index 3ba5a8061..021388642 100644
--- a/configure.ac
+++ b/configure.ac
@@ -536,6 +536,7 @@ dnl check for timing functions
 AC_CHECK_HEADERS([sys/time.h])
 AC_CHECK_FUNCS([getrusage gettimeofday clock_gettime clock_getres])
 AC_CHECK_FUNCS([setitimer])
+AC_CHECK_LIB([rt], [timer_create])
 
 dnl check for functions dealing with virtual memory
 AC_CHECK_FUNCS([vm_allocate sbrk madvise sysconf])

Does this make any kind of sense? I'm currently setting up a Semigroups Travis build to test this.

EDIT: I can confirm that this does fix it.

[james-d-mitchell]

So, if I manually add -pthread to CXXFLAGS this also fixes it in Debian. @fingolfin do you know the correct way to get autoconf/automake to include the flag -pthread when we are compiling with gcc?
I did this in a hacky way in libsemigroups, which I just tried here, but it didn't seem to work. Note that when compiling with clang including -pthread gives a warning that the flag is not necessary, so ideally we'd only add the flag when compiling with gcc. Or perhaps there is an altogether other way of doing this?

[fingolfin]

https://github.com/gap-system/gap/blob/master/cnf/m4/ax_pthread.m4

[james-d-mitchell]

Thanks @fingolfin, but this doesn't seem to do anything, or I am not using it correctly.

I put AX_PTHREAD() into configure.ac and then put:

@echo "PTHREAD_CFLAGS = $(PTHREAD_CFLAGS)"

under the all-local make target in Makefile.am, and it seems that PTHREADS_CFLAGS is always undefined (or rather nothing is echoed after PTHREAD_FLAGS =). Any suggestions would be very gratefully accepted.

[fingolfin]

We use it in GAP, of course, so you can look there for an example of how we use it; namely we just have AX_PTHREAD near the end of configure.ac (no need for parentheses), then in the Makefiles we do (spread across two files):

...
PTHREAD_CFLAGS = @PTHREAD_CFLAGS@
PTHREAD_LIBS = @PTHREAD_LIBS@
...
ifeq ($(HPCGAP),yes)
GAP_CFLAGS += $(PTHREAD_CFLAGS)
endif
...
ifeq ($(HPCGAP),yes)
GAP_LDFLAGS += $(PTHREAD_CFLAGS) $(PTHREAD_LIBS)
endif
...

Did you verify that the AC_PTHREAD macro gets substituted correctly, i.e., by looking at configure? (This can fail if the .m4 file is not found by autoconf resp. autoreconf).

[james-d-mitchell]

Ah thanks @fingolfin, the macro was not getting substituted correctly. Now it is, but I still can't get this to work. I added ax_pthread.m4 to the m4 directory and it is substituted and PTHREAD_CFLAGS is set properly, I tried adding PTHREAD_CFLAGS to AM_CXXFLAGS, semigroups_la_CXXFLAGS, and CXXFLAGS in Makefile.am (separately and in all combinations) but this did not work on Debian. Interestingly it works with clang which complains that the -pthread option is not used, but it otherwise works ok. If I do export CXXFLAGS="-pthread", then configure; make everything works just fine. I am still missing something here...

[james-d-mitchell]

I think I have a fix

[james-d-mitchell]

So, it does appear that I have fixed this, the approach of using AX_PTHREAD, unfortunately did not work, I don't understand why, as it seemed like it should.

[mtorpey]

Fixed by James's PR.