msolve: more platforms, enable OpenMP (WIP)

[fingolfin]

Trying to help @ederc expand this JLL: the main goal is to enable OpenMP on Linux and add Apple M1 support. However, to get started, I also enabled all platforms, just to see what all breaks, and how ;-). As such, this is of course just a draft for now.

[fingolfin]

Unfortunately FLINT_jll lacks an M1 binary so I first have to take care of that...

UPDATE: see PR #3779

[fingolfin]

Huh, fatal error: gmp.h: No such file or directory on msolve-x86_64-linux-musl and other musl platforms, and also on msolve-x86_64-unknown-freebsd... I wonder what might be causing that?! @ederc mentioned that muslc and freebsd are not currently supported by msolve, which is fine, but that doesn't explain why it can't find gmp.h?

[fingolfin]

Fixed the "gmp.h" errors by setting CPPFLAGS (this might hint at an issue with the msolve build system, though, doesn't it?)

[fingolfin]

@ederc you might want to apply this patch to your code

[ederc]

@fingolfin Thanks, done.
I still try to understand your CPPFLAGS solution for gmp.h. msolve is a plain C project and I cannot see where these flags are set at all.

[benlorenz]

Something weird is going on with the CPU-detection, on x86_64-linux-gnu the log contains:

[21:32:13] checking for x86 cpuid  output... unknown
[21:32:13] checking for x86-AVX xgetbv  output... unknown
[21:32:13] checking for x86 cpuid 0x00000000 output... unknown
[21:32:13] checking for x86 cpuid 0x80000000 output... unknown
[21:32:14] checking whether RDRND is supported by the processor... no
[21:32:14] checking whether BMI1 is supported by the processor... no
[21:32:14] checking whether BMI2 is supported by the processor... no
[21:32:14] checking whether ADX is supported by the processor... no
[21:32:14] checking whether PREFETCHWT1 is supported by the processor... no
[21:32:14] checking whether ABM is supported by the processor... no
[21:32:14] checking whether MMX is supported by the processor... no
[21:32:14] checking whether SSE is supported by the processor... no
[21:32:14] checking whether SSE2 is supported by the processor... no
[21:32:14] checking whether SSE3 is supported by the processor... no
[21:32:14] checking whether SSSE3 is supported by the processor... no
[21:32:14] checking whether SSE4.1 is supported by the processor... no
[21:32:14] checking whether SSE4.2 is supported by the processor... no
[21:32:14] checking whether SSE4a is supported by the processor... no
[21:32:14] checking whether SHA is supported by the processor... no
[21:32:14] checking whether AES is supported by the processor... no
[21:32:14] checking whether AVX is supported by the processor... no
[21:32:14] checking whether FMA3 is supported by the processor... no
[21:32:14] checking whether FMA4 is supported by the processor... no
[21:32:14] checking whether XOP is supported by the processor... no
[21:32:14] checking whether AVX2 is supported by the processor... no

but on x86_64-linux-musl there is

[22:04:45] checking whether C compiler accepts -mssse3... yes
[22:04:45] checking whether SSE4.1 is supported by the processor... yes
[22:04:45] checking whether SSE4.1 is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -msse4.1... yes
[22:04:45] checking whether SSE4.2 is supported by the processor... yes
[22:04:45] checking whether SSE4.2 is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -msse4.2... yes
[22:04:45] checking whether SSE4a is supported by the processor... yes
[22:04:45] checking whether SSE4a is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -msse4a... yes
[22:04:45] checking whether SHA is supported by the processor... yes
[22:04:45] checking whether SHA is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -msha... yes
[22:04:45] checking whether AES is supported by the processor... yes
[22:04:45] checking whether AES is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -maes... yes
[22:04:45] checking whether AVX is supported by the processor... yes
[22:04:45] checking whether AVX is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -mavx... yes
[22:04:45] checking whether FMA3 is supported by the processor... yes
[22:04:45] checking whether FMA3 is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -mfma... yes
[22:04:45] checking whether FMA4 is supported by the processor... no
[22:04:45] checking whether XOP is supported by the processor... no
[22:04:45] checking whether AVX2 is supported by the processor... yes
[22:04:45] checking whether AVX2 is supported by the processor and OS... yes
[22:04:45] checking whether C compiler accepts -mavx2... yes

which is then used during the compilation and at the end the auditor complains:

┌ Warning: Minimum instruction set detected for lib/libmsolve-0.1.2.so is avx2, not x86_64 as desired.
└ @ BinaryBuilder.Auditor /depot/packages/BinaryBuilder/BlmHn/src/Auditor.jl:290
##[warning]Minimum instruction set detected for lib/libmsolve-0.1.2.so is avx2, not x86_64 as desired.

Maybe the checks only succeed when run natively on musl but fail everywhere else and then default to no?
So I think that AX_EXT macro needs to be disabled or we might be able to override the CPUEXT_FLAGS and SIMD_FLAGS. But I am not that well versed in autoconf magic so there might be other ways to achieve this...

[giordano]

We should be able to run foreign executables for x86_64-linux-gnu

[benlorenz]

autoconf seems to skip these checks if it detects that it is cross-compiling ($host != $build), but it seems we can override this check by something like this:

echo cross_compiling=yes > $prefix/share/config.site

Which might even make sense to set globally for all autoconf based packages, but could also break some build scripts...

[giordano]

@fingolfin good to go?

[fingolfin]

@giordano I think @ederc should decide that, it's his package, I made this PR to help him.

The issues @benlorenz pointed out should be resolved, too, but that could be done in a future PR: in general, @ederc should think carefully about how to handle this kind of feature detection, given that we are building binaries here that are too be used on many different platforms, and so e.g. building a binary using AVX2 might not be the best idea, unless the plan really is to restrict to systems supporting it.

Then again, I am not sure the results of those tests are all used? If they aren't, perhaps they could be disabled for now?

[ederc]

From my point of view we could take this PR now.

About the feature detection: This is crucial when it comes to performance, for example, AVX2 enabled gives you a factor of 3. Isn't there a way to provide binaries with AVX2 and without AVX2 for all supported platforms?

[fingolfin]

Yes one can do that, using "microarchitectures". Indeed there is a function expand_microarchitectures -- but I see nothing in Yggdrasil actually use it? It might be too broad though, we maybe would only want to define some of the microarchs (e.g. if you have no special code for ARM Neon, no need to have a separate binary for that).

[fingolfin]

I am not quite sure how to use microarchs corectly, though. Maybe @giordano or @staticfloat can advise?

[giordano]

Yes, I don't think there is any package currently using microarchitectures, but this looks like a potentially perfect use case. However this also means we haven't tested that into the wild, so there might be some things to iron out.

I'm keeping a list of packages that could benefit from it in #2209, but the reason why it isn't much used it is because most libraries actually do runtime detection of CPU features (using for example CPUID), which is also more cross-compilation-friendly.

[ederc]

Can we take this PR now? I think the other points we discussed need to be resolved in some future PRs.

[fingolfin]

Yes, please

[giordano]

Uhm, this is changing the compat with FLINT_jll, I believe the registry won't like this within a build version. Is there a newer version of msolve to build to make this transition smoother?

[ederc]

Yes, we can bump the version to 0.1.5 using msolve commit 42137aaa68ac6077d25f7cb8e72ff01ac6f14f41 .

@fingolfin This commit of msolve already includes the changes provided by your sort_r.patch, so I think we can remove the atomic_patch part, right?