Cross-Compile dynamic linked issues

[jayhuang75]

Hello @fede1024,

I need some or guidance, I'm currently trying to compile/build targets "x86_64-unknown-linux-musl"
cargo build --target x86_64-unknown-linux-musl --release --features vendored
and just for clairification the normal build without any problem, and I have the librdkafka installed.

The error keeps point to something I would say relatively with the dynamically linked issues:
`_error: failed to run custom build command for rdkafka-sys v1.3.1

Caused by:
process didn't exit successfully: /home/w/rust/rust-grpc-tonic/target/release/build/rdkafka-sys-b8f266a3d2a85c7e/build-script-build(exit code: 1)
--- stderr
librdkafka will be linked dynamically
librdkafka 1.3.1 cannot be found on the system
Dynamic linking failed. Exiting._`

I have tried this couple day now, could you kindly point me to some right direction. And some of the suggestions I have tried also in the issues list, which still not working.

Thank you in advance for the help here. Appreciated.

[benesch]

To dynamically link while cross compiling requires that you have built and installed a cross-compiled shared version of librdkafka. I wouldn’t be surprised if the build script didn’t handle this use case, actually. Is there any reason you can’t link librdkafka statically? Typically if you’re linking with musl it’s because you want an entirely static binary.

…

On Sun, Jun 7, 2020 at 11:23 AM jayhuang75 ***@***.***> wrote: Hello @fede1024 <https://github.com/fede1024>, I need some or guidance, I'm currently trying to compile/build targets "x86_64-unknown-linux-musl" cargo build --target x86_64-unknown-linux-musl --release --features vendored and just for clairification the normal build without any problem, and I have the librdkafka installed. The error keeps point to something I would say relatively with the dynamically linked issues: `_error: failed to run custom build command for rdkafka-sys v1.3.1 Caused by: process didn't exit successfully: /home/w/rust/rust-grpc-tonic/target/release/build/rdkafka-sys-b8f266a3d2a85c7e/build-script-build(exit code: 1) --- stderr librdkafka will be linked dynamically librdkafka 1.3.1 cannot be found on the system Dynamic linking failed. Exiting._` I have tried this couple day now, could you kindly point me to some right direction. And some of the suggestions I have tried also in the issues list, which still not working. Thank you in advance for the help here. Appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#269>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGXSIBJFA56LL5ZZ3GMGPTRVOWG5ANCNFSM4NXKYQKQ> .

[jayhuang75]

Thank you so much for the quick response, the main reason: we want to deploy to the Alpine and reduce the size for the IoT.

JUST FYI when we try to simply build without MUSL and deploy to the debian:blueye-slim, it is around 400MB, which is working perfectly.

And back to your guidance: is it the solution will download the librdkafka, and cross-compiled it then linked with the build? Or something else I misunderstand?

Really appreciated your help here.

[SnoozeTime]

@jayhuang75
We were able to build a really small alpine image that uses rust-rdkafka, with the ssl feature and without the sasl feature. That means you cannot use advanced sasl features but you can still use some authentication schemes such as PLAIN that are provided in rust-rdkafka.

The trick is to compile with this docker image: https://github.com/clux/muslrust
It has already zlib and openssl installed as static libs.

The binary can then be copied to an alpine image which need to have the variables OPENSSL_STATIC=true, LIBZ_SYS_STATIC=1. The resulting image is less that 10Mb.

Hope that helps

[benesch]

Sorry for the delay in response @jayhuang75. From your initial issue, you can see that librdkafka is getting linked dynamically:

--- stderr
librdkafka will be linked dynamically
librdkafka 1.3.1 cannot be found on the system

That indicates that that something, somewhere, is enabling the dynamic-linking feature. As I mentioned in my last comment, cross-compiling with dynamic linking is much harder to get right. If you don't intend to link dynamically, I suggest you track down what is enabling rdkafka's dynamic-linking feature, and remove it. (Any dependency that links against librdkafka can enable a feature, as features are additive.)

[benesch]

Closing due to inactivity. Feel free to open a new issue if you are still having trouble!