GSL not finding BLAS in Fedora

[gtheler]

Dear @vitorvas

PETSc is found on your Fedora, so configure.ac does not link against gslclbas, see https://github.com/seamplex/feenox/blob/main/configure.ac#L314

I think the problem is that Fedora's GSL is not able to use PETSc's BLAS. Do you have any other insight about what BLAS PETSc is using and why GSL is not finding it?

If we cannot figure out a way to detect this automatically I'll have to add an option to force linking against gslcblas.
What do you think?

[vitorvas]

Dear @gtheler,

I am glad to help, but I'll need your guidance since I have no idea on how PETSC and BLAS are related, nor the implications to GSL.

I'll give you a brief description of the system I first tested feenox:

• PETSC and SLEPC installed from source in /home/vitors/bin/petsc-3.14.1 and /home/vitors/bin/slepc-3.14.0 respectively.

• /home/vitors/bin added to $PATH

The reported problem concerns this system. The reason to have PETSC and SLEPC installed from source is because either SLEPC is not provided by Fedora as a package and because milonga had problems detecting PETSC from the package version. I lived happily with this situation. And here I answer your first question: I don know! I don't even know how PETSC interacts with BLAS. My only guess is that having PETSC installed from source and GSL properly installed as a package somehow doesn't make sense as a properly installed system and that's the reason causing configure.ac to fail. The system in which it is running is messy.

After reading your message, as a very simple test, I just removed my version of PETSC from source and installed PETSC from Fedora package system. I also installed PETSC-devel (the development package in Fedora) Now I have:

[root@cochon feenox]# rpm -q petsc
petsc-3.14.4-1.fc34.x86_64

[root@cochon feenox]# rpm -q petsc-devel
petsc-devel-3.14.4-1.fc34.x86_64

In detail (I'll spare you of the contents of petsc-devel. Let me know if you want it):

[root@cochon feenox]# rpm -ql petsc
/usr/lib/.build-id
/usr/lib/.build-id/92
/usr/lib/.build-id/92/7ac4a74fbdd7d920b67e493480144dedc60778
/usr/lib64/libpetsc.so.3.14
/usr/lib64/libpetsc.so.3.14.4
/usr/share/licenses/petsc
/usr/share/licenses/petsc/LICENSE

Now, feenox installs with charm. Right?

## ----------------------- ##
## Summary of dependencies ##
## ----------------------- ##
GNU Scientific Library from system
SUNDIALS yes
PETSc no
SLEPc no

Wrong. It didn't find the PETSC in Fedora when installed from package.

checking for PETSc dir... no

Obviously this is not a problem for anybody wanting to use feenox. It compiles and builds fine. However, if someone really needs PETSC for some reason, they won't have it for Fedora.

The libpetsc.so installed as a package on my Fedora box has the following dependencies:

[vitors@cochon feenox]$ ldd -r /usr/lib64/libpetsc.so.3.14

linux-vdso.so.1 (0x00007ffc0a9f7000)
libsuperlu.so.5.2 => /lib64/libsuperlu.so.5.2 (0x00007f25f603e000)
libflexiblas.so.3 => /lib64/libflexiblas.so.3 (0x00007f25f5c8c000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f25f5c6b000)
libX11.so.6 => /lib64/libX11.so.6 (0x00007f25f5b23000)
libcgns.so.4.1 => /lib64/libcgns.so.4.1 (0x00007f25f5a44000)
libhdf5.so.103 => /lib64/libhdf5.so.103 (0x00007f25f56c1000)
libm.so.6 => /lib64/libm.so.6 (0x00007f25f557b000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f25f5574000)
libc.so.6 => /lib64/libc.so.6 (0x00007f25f53a5000)
libgfortran.so.5 => /lib64/libgfortran.so.5 (0x00007f25f50f8000)
libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f25f50ae000)
/lib64/ld-linux-x86-64.so.2 (0x00007f25f73e4000)
libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f25f5083000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f25f5066000)
libsz.so.2 => /lib64/libsz.so.2 (0x00007f25f505c000)
libz.so.1 => /lib64/libz.so.1 (0x00007f25f5042000)
libXau.so.6 => /lib64/libXau.so.6 (0x00007f25f503c000)`

I only see a libflexiblas to blame for providing BLAS funcionality to PETSC.

So, finally getting to your last question, I think the best solution is to make sure the configure.ac is capable of finding the "natural" installation of PETSC for Fedora systems . I guess if it finds the packager version of PETSC, there will be no issues with GSL.

[gtheler]

Can you tell me the absolute path of the include directory of petsc-devel?
This is what I get grepping for blas in the PETSc provided by Debian:

gtheler@tom:/usr/lib/petsc/lib$ ldd libpetsc_real.so.3.14.5 | grep blas
        libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007f0363f66000)
        libopenblas.so.0 => /lib/x86_64-linux-gnu/libopenblas.so.0 (0x00007f0360d47000)
gtheler@tom:/usr/lib/petsc/lib$ 

And I get the same for a hand-compiled PETSc. Now

gtheler@tom:/usr/lib/petsc/lib$ nm -D /lib/x86_64-linux-gnu/libblas.so.3 | grep dgemm | head -n1
00000000000318d0 T cblas_dgemm
gtheler@tom:/usr/lib/petsc/lib$ nm -D /lib/x86_64-linux-gnu/libblas.so.3 | grep dgemm | head -n1
00000000000318d0 T cblas_dgemm
gtheler@tom:/usr/lib/petsc/lib$ 

So both of them export the cblas_dgemm symbol which is what configure.ac needs in line 315.
My hunch is that this libflexiblas exports dgemm but not cblas_dgemm. Can you check?

[vitorvas]

Sure. It is: /usr/include/petsc.

And checking libflexiblas, I got the symbol we're looking for:

[vitors@cochon conf]$ nm -D /usr/lib64/libflexiblas.so.3.0 | grep dgemm | head -n1
0000000000136d70 T cblas_dgemm

Additionally, I tried to learn where this flexiblas comes from and it seems Fedora favors this "benchmark-able" approach here.

So, with flexiblas providing cblas_dgemm the symbol is there. However, the problem continues: since the package version of PETSC for Fedora does not adds the $PETSC_DIR nor $PETSC_ARCH, we'll not be able to find PETSC in Fedora. Is this the reason you asked for the include file, right? I'm trying to follow you but configure.ac is an alien to me. I'm used to CMake.

[gtheler]

AFAIK just adding -lpetsc should also add all the linked libraries so the symbol cblas_dgemm should be autoamtically available.
Not sure what's going on.

I asked for the include directory because that's one of the things that configure searches for. But apparently Debian works differently so the logic in configure.ac won't work on Fedora.

In any case, I moved the discussion to issues. See #5 and #6
For now I'll always link against gslcblas to partially fix the compilation problem. See 0da38bb