build: cmake: need to uniformize installation of libraries in CMAKE_INSTALL_LIBDIR (not lib)

[zougloub]

🐛 Describe the bug

Installed pytorch from source, with cmake.

In particular I noticed that some libraries are installed in a hard-coded $DESTDIR/$CMAKE_INSTALL_PREFIX/lib, which is not expected on x86_64 (should be .../lib64 from CMAKE_INSTALL_LIBDIR).

Looking at the cmake files:

git grep CMAKE_INSTALL_LIBDIR
git grep "DESTINATION lib"

and found issues.

So I propose to do this:

git grep " lib)" | perl -pe 's/(.*):(.*)$\s*/\1/g' | grep CMakeLists.txt | uniq | while read file; do sed -i -e 's@ lib)@ ${CMAKE_INSTALL_LIBDIR})@' "$file"; done

then reviewing with git gui or something else, to get rid of false positives (eg. test executables shouldn't go there, and it's OK for the deployed platform-agnostic .py files to be in .../lib).

Versions

PyTorch version: 1.13.0+cu117

cc @malfet @seemethere @jbschlosser

[zougloub]

Add torch/lib/libshm/CMakeLists.txt currently doing:

set(LIBSHM_INSTALL_LIB_SUBDIR "lib" CACHE PATH "libshm install library directory")

[malfet]

which is not expected on x86_64 (should be .../lib64 from CMAKE_INSTALL_LIBDIR

Can you please share a reference to some sort of standard requiring the 64-binaries must be put in lib64 folder? In an early day of x86_64 and multiarch distros it was indeed the case, but I think now trend is opposite: i686 shared libraries would be installed in lib32 folder, while 64-bit ones are installed in /usr/lib/.
For example on 64-bit Ubuntu system:

$ file /usr/lib/libmunge.so.2.0.0 
/usr/lib/libmunge.so.2.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=7265decb1fc98bcdd197ca9ff7290192127e8b8e, with debug_info, not stripped

[zougloub]

CMAKE_INSTALL_LIBDIR=lib64 was a particular case, and there is no standard requiring lib64, the point was that it could be lib or something else, hence CMAKE_INSTALL_LIBDIR should be used instead of hard-coded lib (it's probably its raison d'être as a CMAKE_* variable).

• Fedora x86_64 (https://docs.fedoraproject.org/en-US/packaging-guidelines/) uses lib64:

gpg => /usr/bin/gpg (interpreter => /lib64/ld-linux-x86-64.so.2)
    libz.so.1 => /lib64/libz.so.1
    libbz2.so.1 => /lib64/libbz2.so.1
    libsqlite3.so.0 => /lib64/libsqlite3.so.0
        libm.so.6 => /lib64/libm.so.6
            ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
    libgcrypt.so.20 => /lib64/libgcrypt.so.20
    libreadline.so.8 => /lib64/libreadline.so.8
        libtinfo.so.6 => /lib64/libtinfo.so.6
    libassuan.so.0 => /lib64/libassuan.so.0
    libnpth.so.0 => /lib64/libnpth.so.0
    libgpg-error.so.0 => /lib64/libgpg-error.so.0
    libc.so.6 => /lib64/libc.so.6

• Debian & Ubuntu use subdirectories -> https://wiki.ubuntu.com/MultiarchSpec

/usr/bin/gpg (interpreter => /lib64/ld-linux-x86-64.so.2)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0
    libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20
    libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
    libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8
        libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6
    libassuan.so.0 => /lib/x86_64-linux-gnu/libassuan.so.0
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6

• Gentoo uses lib64 -> https://wiki.gentoo.org/wiki/Project:AMD64/Multilib_layout

gpg => /usr/bin/gpg (interpreter => /lib64/ld-linux-x86-64.so.2)
    libz.so.1 => /lib64/libz.so.1
    libbz2.so.1 => /lib64/libbz2.so.1
    libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0
        libm.so.6 => /lib64/libm.so.6
            ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
        libicui18n.so.72 => /usr/lib64/libicui18n.so.72
            libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/12/libstdc++.so.6
            libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/12/libgcc_s.so.1
        libicuuc.so.72 => /usr/lib64/libicuuc.so.72
            libicudata.so.72 => /usr/lib64/libicudata.so.72
    libgcrypt.so.20 => /usr/lib64/libgcrypt.so.20
    libgpg-error.so.0 => /usr/lib64/libgpg-error.so.0
    libreadline.so.8 => /lib64/libreadline.so.8
        libtinfow.so.6 => /lib64/libtinfow.so.6
    libassuan.so.0 => /usr/lib64/libassuan.so.0
    libnpth.so.0 => /usr/lib64/libnpth.so.0
    libc.so.6 => /lib64/libc.so.6

[malfet]

I see your point. Please keep in mind though, that when PyTorch is build, those libs are installed inside torch/ folder in the sidepackages, rather than in system installed folder. But I get your point, that it should be different, if BUILD_PYTHON is set to false (i.e. libtorch is installed) Please do not hesitate to propose the PR to address this issue.