Fix NTL spkg-configure.m4 so it rejects NTLs built with NTL_THREADS

[mkoeppe]

... such as homebrew's NTL (#29104).

Builds with NTL_THREADS breaks Singular (as observed in #29104).

In fact, as of the NTL upgrade ticket #20590, we build the NTL SPKG with NTL_THREADS=off. (On that ticket, it was noted "we cannot take advantage of the threading until a number of things are resolved. Threading requires C++11 and a number of packages are behind the curve.")

In a follow-up ticket we may consider issuing a warning if NTL is configured without NTL_GMP_LIP, without NTL_GF2X_LIB, which may lead to a performance degradation.

CC: @dimpase @kiwifb @mwageringel @jhpalmieri @jpflori

Component: build: configure

Branch/Commit: u/mkoeppe/fix_ntl_spkg_configure_m4_so_it_rejects_ntls_built_with_ntl_threads__without_ntl_gmp_lip__without_ntl_gf2x_lib @ c3bc092

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29339

[dimpase]

comment:1

the wise thing is to enable TLS as much as possible, IMHO. The configuration with enabled TLS on NTL, pari, and flint used to work, IIRC.

[dimpase]

comment:2

I also don't think gf2x should be a hard requirement. It is entirely optional for NTL.

[mkoeppe]

comment:3

Do you mean NTL_TLS_HACK?

[mkoeppe]

comment:4

homebrew's NTL is configured with NTL_TLS_HACK and NTL_THREADS, and I am getting the reported error with Singular.
What do you suggest?

[mkoeppe]

comment:5

Tests run at https://github.com/mkoeppe/sage/actions/runs/56215147

[dimpase]

comment:7

see #27764

[mkoeppe]

Branch: u/mkoeppe/fix_ntl_spkg_configure_m4_so_it_rejects_ntls_built_with_ntl_threads__without_ntl_gmp_lip__without_ntl_gf2x_lib

[mkoeppe]

Commit: 96553db

[mkoeppe]

New commits:

96553db

build/pkgs/ntl/spkg-configure.m4: Check for NTL configuration

[mkoeppe]

Author: Matthias Koeppe

[dimpase]

comment:10

once again - some distros build NTL without gf2x support, and it results in some deterioration of performance, but this is not something Sage should worry too much. Print a warning if you must.

After all, it is by far not the only place where the performance is suboptimal on pre-build installs, untuned for the hardware, say.

[sagetrac-git]

Changed commit from 96553db to c3bc092

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c3bc092

Remove tests for NTL_GMP_LIP, NTL_GF2X_LIB

[mkoeppe]

comment:14

Replying to @dimpase:

once again - some distros build NTL without gf2x support, and it results in some deterioration of performance, but this is not something Sage should worry too much. Print a warning if you must.

OK, I have removed the tests

[dimpase]

comment:15

IMHO also pari/gp is typically built with threads on by distros.
This means that TLS is enabled for pari/gp.

We may end up with having to build all the libs...

[mkoeppe]

comment:16

I don't see such problems in my tests

[dimpase]

comment:17

do you test with options forcing the use of system packages?

[mkoeppe]

comment:18

Not recently, will do

[mkoeppe]

comment:25

Shouldn't there be a ticket that enables NTL threads for Sage-the-distribution then?
The NTL upgrade ticket #20590 disabled threads explicitly.
So I assume there was a difficulty -- and there is no evidence that it has been resolved.

[dimpase]

comment:26

4 years ago the Python's single-thread religion still had many followers :-)

[mkoeppe]

comment:27

No, seriously, when our own installation scripts avoid NTL's threads, then I think it is quite appropriate that the spkg-configure checks for that.

[dimpase]

comment:28

by the way, Fedora 30 (and on newer Fedoras too) also provides multithreaded NTL, and Sage happily uses it, tests pass.

$ uname -a
Linux clpc171.cs.ox.ac.uk 5.2.18-200.fc30.x86_64 #1 SMP Tue Oct 1 13:14:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ ldd /lib64/libntl.so
        linux-vdso.so.1 (0x00007ffe9affd000)
        libgmp.so.10 => /lib64/libgmp.so.10 (0x00007fabe7899000)
        libgf2x.so.1 => /lib64/libgf2x.so.1 (0x00007fabe7888000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fabe7867000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fabe766d000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fabe7527000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fabe7361000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fabe7c33000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fabe7345000)

[dimpase]

comment:29

Replying to @mkoeppe:

No, seriously, when our own installation scripts avoid NTL's threads, then I think it is quite appropriate that the spkg-configure checks for that.

I don't see why. Anyhow, this problem (inability to use multithreaded NTL) seems to be MacOS-only, and probably related to #27764

[kiwifb]

comment:30

We historically had an issue before defaulting to C++11 but we should be able to move ahead in my opinion. I am expecting distros to all be threaded at this stage. It is just a matter of testing the insane number of corner cases.

[dimpase]

comment:31

Replying to @kiwifb:

We historically had an issue before defaulting to C++11 but we should be able to move ahead in my opinion. I am expecting distros to all be threaded at this stage. It is just a matter of testing the insane number of corner cases.

it seems to be that Debian is not providing multithreaded NTL (and the related libs, such as Flint, are single-threaded too).

[mwageringel]

comment:32

For what it is worth, on macOS 10.13.6, Homebrew's NTL works for me without any problems. I last checked with a clean build of 9.1.beta4 and ptestlong passed.

[mkoeppe]

comment:33

Replying to @kiwifb:

We historically had an issue before defaulting to C++11 but we should be able to move ahead in my opinion. I am expecting distros to all be threaded at this stage. It is just a matter of testing the insane number of corner cases.

I have created ticket #29340 (Enable threads in NTL) for that, in case someone would like to work on it.

[mkoeppe]

comment:34

Replying to @dimpase:

Replying to @mkoeppe:

No, seriously, when our own installation scripts avoid NTL's threads, then I think it is quite appropriate that the spkg-configure checks for that.

I don't see why. Anyhow, this problem (inability to use multithreaded NTL) seems to be MacOS-only, and probably related to #27764

#27764 does not fix the problem on macOS, I've already tested that

[mkoeppe]

comment:35

See #29366 (arch)

[mkoeppe]

Changed author from Matthias Koeppe to none

[mkoeppe]

comment:38

I think this is no longer needed

[dimpase]

Reviewer: Dima Pasechnik

[dimpase]

comment:39

ok