Build fails on RHEL8

[cyqsimon]

Version

0.3.1

Description

cargo install ouch errors during compilation. See full log down bottom.

I took a look at the error and it seems like external functions of zstd are not found. So I ran dnf install zstd libzstd libzstd-devel but all is already installed and up to date. So I don't know where to go from here.

Env:

RHEL8 Linux 4.18.0-348.12.2.el8_5.x86_64

Full log:

   Compiling ouch v0.3.1
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/tmp/cargo-installr9gAFi/release/deps" "-L" "/tmp/cargo-installr9gAFi/release/build/bzip2-sys-3d0b37a7f4b075e1/out/lib" "-L" "/usr/lib64" "-L" "/tmp/cargo-installr9gAFi/release/build/zstd-sys-9e622936af957d1d/out" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustc4bCbwn/libzstd_sys-777a7a39a9bef298.rlib" "/tmp/rustc4bCbwn/libbzip2_sys-8140410cc95a8a3a.rlib" "-Wl,--start-group" "-Wl,--end-group" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-1d9f7e5920635d15.rlib" "-Wl,-Bdynamic" "-llzma" "-lz" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-Wl,-O1" "-nodefaultlibs"
  = note: /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `ouch::commands::compress_files::{{closure}}':
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x25c): undefined reference to `ZSTD_createCCtx'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x26f): undefined reference to `ZSTD_CCtx_setParameter'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x27b): undefined reference to `ZSTD_isError'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x295): undefined reference to `ZSTD_CCtx_loadDictionary'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x2a1): undefined reference to `ZSTD_isError'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x543): undefined reference to `ZSTD_freeCCtx'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands14compress_files28_$u7b$$u7b$closure$u7d$$u7d$17h094feb598bf79cdbE+0x719): undefined reference to `ZSTD_freeCCtx'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `core::ptr::drop_in_place$LT$zstd..stream..write..AutoFinishEncoder$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$alloc..boxed..Box$LT$dyn$u20$core..ops..function..FnMut$LT$$LP$core..result..Result$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$std..io..error..Error$GT$$C$$RP$$GT$$u2b$Output$u20$$u3d$$u20$$LP$$RP$$u2b$core..marker..Send$GT$$GT$$GT$::hf1793bba825767e8':
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr351drop_in_place$LT$zstd..stream..write..AutoFinishEncoder$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$alloc..boxed..Box$LT$dyn$u20$core..ops..function..FnMut$LT$$LP$core..result..Result$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$std..io..error..Error$GT$$C$$RP$$GT$$u2b$Output$u20$$u3d$$u20$$LP$$RP$$u2b$core..marker..Send$GT$$GT$$GT$17hf1793bba825767e8E+0x6f): undefined reference to `ZSTD_endStream'
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr351drop_in_place$LT$zstd..stream..write..AutoFinishEncoder$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$alloc..boxed..Box$LT$dyn$u20$core..ops..function..FnMut$LT$$LP$core..result..Result$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$std..io..error..Error$GT$$C$$RP$$GT$$u2b$Output$u20$$u3d$$u20$$LP$$RP$$u2b$core..marker..Send$GT$$GT$$GT$17hf1793bba825767e8E+0x76): undefined reference to `ZSTD_isError'
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr351drop_in_place$LT$zstd..stream..write..AutoFinishEncoder$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$alloc..boxed..Box$LT$dyn$u20$core..ops..function..FnMut$LT$$LP$core..result..Result$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$std..io..error..Error$GT$$C$$RP$$GT$$u2b$Output$u20$$u3d$$u20$$LP$$RP$$u2b$core..marker..Send$GT$$GT$$GT$17hf1793bba825767e8E+0x1fe): undefined reference to `ZSTD_freeCCtx'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `<zstd::stream::write::AutoFinishEncoder<W,F> as std::io::Write>::write':
          ouch.fdb27b3a-cgu.0:(.text._ZN86_$LT$zstd..stream..write..AutoFinishEncoder$LT$W$C$F$GT$$u20$as$u20$std..io..Write$GT$5write17h78948c6f603f447fE+0x51): undefined reference to `ZSTD_CCtx_reset'
          ouch.fdb27b3a-cgu.0:(.text._ZN86_$LT$zstd..stream..write..AutoFinishEncoder$LT$W$C$F$GT$$u20$as$u20$std..io..Write$GT$5write17h78948c6f603f447fE+0x5d): undefined reference to `ZSTD_isError'
          ouch.fdb27b3a-cgu.0:(.text._ZN86_$LT$zstd..stream..write..AutoFinishEncoder$LT$W$C$F$GT$$u20$as$u20$std..io..Write$GT$5write17h78948c6f603f447fE+0xeb): undefined reference to `ZSTD_compressStream'
          ouch.fdb27b3a-cgu.0:(.text._ZN86_$LT$zstd..stream..write..AutoFinishEncoder$LT$W$C$F$GT$$u20$as$u20$std..io..Write$GT$5write17h78948c6f603f447fE+0xf7): undefined reference to `ZSTD_isError'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `<zstd::stream::write::AutoFinishEncoder<W,F> as std::io::Write>::flush':
          ouch.fdb27b3a-cgu.0:(.text._ZN86_$LT$zstd..stream..write..AutoFinishEncoder$LT$W$C$F$GT$$u20$as$u20$std..io..Write$GT$5flush17hc6d937695db3db2dE+0x29): undefined reference to `ZSTD_flushStream'
          ouch.fdb27b3a-cgu.0:(.text._ZN86_$LT$zstd..stream..write..AutoFinishEncoder$LT$W$C$F$GT$$u20$as$u20$std..io..Write$GT$5flush17hc6d937695db3db2dE+0x30): undefined reference to `ZSTD_isError'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `core::ptr::drop_in_place<zstd::stream::zio::writer::Writer<alloc::boxed::Box<dyn std::io::Write>,zstd::stream::raw::Encoder>>':
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr138drop_in_place$LT$zstd..stream..zio..writer..Writer$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$zstd..stream..raw..Encoder$GT$$GT$17h71ecd195099a9cb1E+0x2a): undefined reference to `ZSTD_freeCCtx'
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr138drop_in_place$LT$zstd..stream..zio..writer..Writer$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Write$GT$$C$zstd..stream..raw..Encoder$GT$$GT$17h71ecd195099a9cb1E+0x75): undefined reference to `ZSTD_freeCCtx'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `ouch::commands::decompress_file::{{closure}}':
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0x21d): undefined reference to `ZSTD_DStreamInSize'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0x318): undefined reference to `ZSTD_createDCtx'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0x324): undefined reference to `ZSTD_initDStream'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0x336): undefined reference to `ZSTD_DCtx_loadDictionary'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0x342): undefined reference to `ZSTD_isError'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0x361): undefined reference to `ZSTD_freeDCtx'
          ouch.fdb27b3a-cgu.0:(.text._ZN4ouch8commands15decompress_file28_$u7b$$u7b$closure$u7d$$u7d$17hcbb5bcbabe911da0E+0xf76): undefined reference to `ZSTD_freeDCtx'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `core::ptr::drop_in_place<zstd::stream::read::Decoder<std::io::buffered::bufreader::BufReader<alloc::boxed::Box<dyn std::io::Read>>>>':
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr149drop_in_place$LT$zstd..stream..read..Decoder$LT$std..io..buffered..bufreader..BufReader$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Read$GT$$GT$$GT$$GT$17hec694670db71e26cE+0x42): undefined reference to `ZSTD_freeDCtx'
          ouch.fdb27b3a-cgu.0:(.text._ZN4core3ptr149drop_in_place$LT$zstd..stream..read..Decoder$LT$std..io..buffered..bufreader..BufReader$LT$alloc..boxed..Box$LT$dyn$u20$std..io..Read$GT$$GT$$GT$$GT$17hec694670db71e26cE+0x74): undefined reference to `ZSTD_freeDCtx'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `<zstd::stream::read::Decoder<R> as std::io::Read>::read':
          ouch.fdb27b3a-cgu.0:(.text._ZN70_$LT$zstd..stream..read..Decoder$LT$R$GT$$u20$as$u20$std..io..Read$GT$4read17hc484f1d9e6785c83E+0x191): undefined reference to `ZSTD_DCtx_reset'
          ouch.fdb27b3a-cgu.0:(.text._ZN70_$LT$zstd..stream..read..Decoder$LT$R$GT$$u20$as$u20$std..io..Read$GT$4read17hc484f1d9e6785c83E+0x19d): undefined reference to `ZSTD_isError'
          ouch.fdb27b3a-cgu.0:(.text._ZN70_$LT$zstd..stream..read..Decoder$LT$R$GT$$u20$as$u20$std..io..Read$GT$4read17hc484f1d9e6785c83E+0x249): undefined reference to `ZSTD_decompressStream'
          ouch.fdb27b3a-cgu.0:(.text._ZN70_$LT$zstd..stream..read..Decoder$LT$R$GT$$u20$as$u20$std..io..Read$GT$4read17hc484f1d9e6785c83E+0x255): undefined reference to `ZSTD_isError'
          /tmp/cargo-installr9gAFi/release/deps/ouch-4c3af697a247f9ad.ouch.fdb27b3a-cgu.0.rcgu.o: In function `zstd::map_error_code':
          ouch.fdb27b3a-cgu.0:(.text._ZN4zstd14map_error_code17h792b0fe890792fb6E+0xd): undefined reference to `ZSTD_getErrorName'
          collect2: error: ld returned 1 exit status
          
  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: failed to compile `ouch v0.3.1`, intermediate artifacts can be found at `/tmp/cargo-installr9gAFi`

Caused by:
  could not compile `ouch` due to previous error

[marcospb19]

Sorry, I do not know the dependencies installation instructions for RHEL8, but to solve this, we are shipping a musl pre-built binary that does not require any dependencies.

You can check it at the releases page https://github.com/ouch-org/ouch/releases.

[cyqsimon]

So I quit being lazy and actually dug in and did some experimenting. Turns out there is a hacky fix, but I'm unsure why.

The issue seems to be that the version of libzstd (and libzstd-devel) packaged for RHEL 8 is too old at v1.4.4. See pkgs.org. Now I'm not sure why there's incompatibility between library v1.4.x and bindings v1.5.y but it is what it is.

Here's the hacky bit. I installed and enabled the newest GCC toolchain available on RHEL 8 (out of desperation more than anything else), and surprisingly ouch successfully compiled with libzstd and libzstd-devel still at v1.4.4. So I proceeded to try other versions of C compilers and here are the results:

Compiler	RHEL 8 Package name	Result
clang 13.0.1	clang	error
gcc 8.5.0	gcc	error
gcc 9.2.1	gcc-toolset-9	error
gcc 10.3.1	gcc-toolset-10	success
gcc 11.2.1	gcc-toolset-11	success

So I guess yay? Anyways, this post is just to share how I solved the issue in case anyone else runs into it. If you have ideas why all this is happening please feel free to share them too.

P.s. It seems like as long as libzstd library is at v1.5.x, any version of gcc as old as v4.8.5 (as is the case on RHEL 7, tested) will compile without problems.

[cyqsimon]

And btw to anyone who intends to install on RHEL, I built it as a COPR package here. You may find it useful.