Compile openblas with openmp

[ViralBShah]

We have been compiling a bunch of libraries in BB with openmp support. Given that we now have all our toolchains reasonably well sorted out, it should be easy for us to build openblas with openmp support.

Does anyone have experience with using openblas with openmp, and if there are any performance issues?

[ViralBShah]

Also, as per the OpenBLAS FAQ, starting 0.3.7, we should be using USE_LOCKING=1 if multiple threads call OpenBLAS and if compiled for single threaded use (USE_THREAD=0).

https://github.com/JuliaPackaging/Yggdrasil/blob/master/O/OpenBLAS/common.jl

We probably should update the build scripts to account for that, but in reality it shouldn't be an issue since we always build multi-threaded openblas.

[ViralBShah]

@giordano @staticfloat @haampie FYI.

[ViralBShah]

This is a bit of a tangent, but I was looking into BLIS.

BLIS is safe with multi-threading: https://github.com/flame/blis/blob/master/docs/Multithreading.md

Also, these appear 6 months old, but BLIS seems to outperform OpenBLAS (0.3.5). The difference on large numbers of cores seems quite a bit. Anyone here with experience? Maybe we should get a BB recipe going for BLIS.
https://github.com/flame/blis/blob/master/docs/Performance.md

[ViralBShah]

Closing as dup of #32786 (sort of)

[chriscoey]

@ViralBShah are you aware of any attempts at using BLIS from Julia? The progress on #32786 seems slow so I'm wondering if Julia needs to look at more alternatives for multithreading and linear algebra

[ViralBShah]

There was an effort a while ago. I should note that it should be straightforward, because BLIS offers a BLAS-compatible API.

OpenBLAS kernels have greatly improved but the multi-threading still needs a lot of work.

The other alterative is to only use the low-level kernels, and build the multi-threaded BLAS and LAPACK in Julia itself. It is not too hard to do such a thing nowadays, but it needs a concerted community effort in that direction.

[ViralBShah]

@chriscoey @YingboMa and others have explored the possibility of a Julia-native BLAS.