can't find libcvmfs.a in the rpms for almalinux9

[btovar]

Probably I'm missing something obvious, but after doing a dnf install cvmfs-devel on almalinux9 I cannot find libcvmfs.a.

From a repoquery:

dnf repoquery -l cvmfs-devel | head -n5
/usr/include/libcvmfs.h
/usr/include/libcvmfs_cache.h
/usr/lib64/libcvmfs.a
/usr/lib64/libcvmfs_cache.a
...etc...

However, inspecting the rpms I don't see it:

rpm2cpio cvmfs-devel-2.11.3-1.el9.x86_64.rpm | cpio -idmv
./usr/include/libcvmfs.h
./usr/include/libcvmfs_cache.h
./usr/share/doc/cvmfs-devel
./usr/share/doc/cvmfs-devel/AUTHORS
./usr/share/doc/cvmfs-devel/COPYING
./usr/share/doc/cvmfs-devel/ChangeLog
./usr/share/doc/cvmfs-devel/README.md
231 blocks

rpm2cpio cvmfs-libs-2.11.3-1.el9.x86_64.rpm | cpio -idmv
./usr/lib/.build-id
./usr/lib/.build-id/76
./usr/lib/.build-id/76/540dba8b68eb85f59bdc3e4670f7ddabe43b78
./usr/lib/.build-id/7e
./usr/lib/.build-id/7e/149435b0d5be2f5d530aa20d73d0c6be898dfb
./usr/lib/.build-id/9a
./usr/lib/.build-id/9a/ee3814396aef2b2e49c1b99cd31cccd04aea22
./usr/lib/.build-id/ba
./usr/lib/.build-id/ba/4f920a4e534a3f218d59a8f0d7a92e54612ab2
./usr/lib/.build-id/cc
./usr/lib/.build-id/cc/3d23e1c5b0ec7462eeb2be9f379a8809aedcfc
./usr/lib/.build-id/f9
./usr/lib/.build-id/f9/636677388bea75fb2ce3c5da39cb035bff2332
./usr/lib64/libcvmfs_cache.so
./usr/lib64/libcvmfs_cache.so.2.11.3
./usr/lib64/libcvmfs_client.so
./usr/lib64/libcvmfs_client.so.2.11.3
./usr/lib64/libcvmfs_crypto.so
./usr/lib64/libcvmfs_crypto.so.2.11.3
./usr/lib64/libcvmfs_crypto_debug.so
./usr/lib64/libcvmfs_crypto_debug.so.2.11.3
./usr/lib64/libcvmfs_util.so
./usr/lib64/libcvmfs_util.so.2.11.3
./usr/lib64/libcvmfs_util_debug.so
./usr/lib64/libcvmfs_util_debug.so.2.11.3
./usr/share/doc/cvmfs-libs
./usr/share/doc/cvmfs-libs/AUTHORS
./usr/share/doc/cvmfs-libs/COPYING
./usr/share/doc/cvmfs-libs/ChangeLog
./usr/share/doc/cvmfs-libs/README.md
73987 blocks

[vvolkl]

Hi! This is as it is intended from our side - we've removed static libraries from cvmfs-devel a while ago, replacing them with the shared library in cvmfs-libs. If there's a particular usecase you need a static library for, I think you'd need to build from source with some small CMake adaptations, but the shared library should work just as well.

I think also our packaging is correct, you may have been misled by dnf repoquery matching all available cvmfs-devel packages (including old ones from before this change). If I do dnf repoquery -l cvmfs-devel-0:2.11.3-1.el9.x86_64 the package contents are displayed correctly.

[btovar]

@vvolkl The use case is parrot, which is used in places where cvmfs has not been installed, therefore the shared library wouldn't be available either. The idea is to have an executable with minimal dynamically linked dependencies to provide cvmfs access.

In fact, as mentioned here: https://cvmfs.readthedocs.io/en/stable/apx-rpms.html, I think that libcvmfs.a was packed in great part to support parrot. Before libcvmfs.a was included, we used to compile cvmfs from source, and as you say, we did have to play with the cmakes file to make it work from version to version, thus being able to directly get it from the rpm is quite advantageous.

[DrDaveD]

@btovar is cvmfsexec an option for you instead of parrot?

[btovar]

Hi Dave, I'm trying to generate the official binaries for parrot. I had not updated the images we use for building them for a while, so I didn't notice that libcvmfs.a was dropped from the rpms.

[vvolkl]

Right, apologies for not warning you about dropping the static libraries! I'm happy to provide these again for the next version, however this will likely require #3576 to avoid a symbol clash from LibreSSL.

[jblomer]

Apologies, that was my bad. Back at the time when we dropped the static library I missed discussing it with you.

The background for dropping the static library is this: we switched to libressl to work around openssl 3 incompatibilities. We use a static built of libressl so that it doesn't clash with openssl, and that static builds is part of the libcvmfs_crypto dynamic library where the libressl symbols are hidden. Making everything part of a single static library and avoid openssl symbol clashes turned out to be difficult.

Now, for the 2.12 release this summer we would like to move to libnettle as the low-level crypto library. When this is done, as Valentin says, we could in principle again build a static libcvmfs. Do I understand correctly that you'd like (more or less) a single relocatable binary for parrot, e.g. linking with rpath to libcvmfs DLLs would not be acceptable?

[btovar]

Valentin, Jakob,

I think I have seen those OpenSSL incompatibilities when trying to statically link from source, and therefore was very convenient to get it from the rpm. The use case for parrot is to land in compute nodes that have not been setup in any way and that may not have the correct namespaces/container stuff enabled for cvmfsexec, so it is more convenient to have everything in a single executable.

As more sites support namespaces/apptainer/etc., the need for parrot is reduced, thus it would not be worth it if it takes a great effort on your part to generate libcvmfs.a. As you mention, one option would be do rpath tricks with the dynamic libraries, however currently parrot is more likely to be supporting older or legacy workflows that have not changed in a while, so the more we can keep things the same the better.

One thing I will try for now is to use cvmfs-devel-0:2.10.1-1.el9, which I think is the last version with libcvmfs.a so I can get a parrot binary for almalinux9. If generating libcvmfs.a gets too messy even after #3576, we can decide with Doug what to do about parrot in the future.

Thanks for all your work!