0.21.0 breaks build on Rust Docker image

[austinhartzheim]

Problem

Using the official Rust docker image, I am unable to build version 0.21.0, whereas 0.20.0 worked without issue. The error I see is:

--- stderr
Building and linking librdkafka statically
Running command: "./configure --disable-sasl --enable-ssl --disable-lz4" in dir: librdkafka
Running command: "make -j 4 libs" in dir: librdkafka
thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'libclang.so\', \'libclang-*.so\', \'libclang.so.*\'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

Solution

I am able to build successfully after installing some additional dependencies in the image (copied from the Dockerfile in this repo):
apt-get install -y llvm-3.9-dev libclang-3.9-dev clang-3.9

Based on the description in d9321f2, it looks like the new dependencies are intended. Perhaps the new dependencies could be mentioned in the changelog to help anyone with broken builds?

Let me know if there is additional information I can provide.

Thanks!

[ArtRand]

I also encountered this, on my Linux machine (macOS was fine). I suspect that anyone using docker to build a project will also see it and scratch their head. A pre-install check or at least a README would be great. Also sounds like other rust projects suffer from this (from googling).

[gertvdijk]

• The configure script passes without clang installed, but fails on make, which is very misleading and I think it the configure script of rdkafka is missing some checks.
• The commit mentions the requirement on libclang, but if you install only libclang packages, the build of rdkafka still fails on a missing stddef.h - you need the clang package too.
• The clang-6.0/clang-7 package from stretch-backports/buster also works.

[thijsc]

I've been spending a bunch of time at getting things to compile because of this today. I see how generating the bindings is easier for the release process, but for users of the crate it's quite a lot of hassle. Think this might even hold back adoption because it's not apparent why you can't build.

I think we should consider rolling this back.

[thijsc]

Did anybody get this to work with the CircleCI Rust image? Even after installing the packages I get:

Building and linking librdkafka statically
Running command: "./configure --disable-sasl --enable-ssl --disable-lz4" in dir: librdkafka
Running command: "make -j 2 libs" in dir: librdkafka
thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'libclang.so\', \'libclang-*.so\', \'libclang.so.*\'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', src/libcore/result.rs:997:5
stack backtrace:

[thijsc]

Ping, still don't have a working build on CircleCI. I will open a pull request to revert the bindgen change.

[bruceg]

I am also unable to get 0.21.0 to compile on Gentoo Linux with llvm/clang 8 without setting $LIBCLANG_PATH. Version 0.20.0 works fine.

[Roguelazer]

Note that this completely breaks 0.21 for platforms that have Rust but no Clang, as far as I can tell... It would be nice to continue to support platforms that don't have clang...

[benesch]

Hi, sorry for the pain that PR caused. Unfortunately I don't have easy access to a FreeBSD machine, so checking in the bindings meant that I couldn't bump the version of librdkafka, since I couldn't regenerate the bindings on FreeBSD. (And this is a cost that must be paid whenever there's a new librdkafka version, which is often.) One thing we could consider is bundling the bindings on Linux only, I suppose.

Note that there's an upstream issue for bindgen to bundle libclang that may be of interest: rust-lang/rust-bindgen#918

[Roguelazer]

Would it be possible to, I dunno, collect money to run an EC2 instance once a quarter when a new release happens? This is a breaking change for me because there's no straightforward way to get an appropriate build of libclang for my platform.

[benesch]

What platform do you use, @Roguelazer?

[Roguelazer]

RHEL6

…

-- James Brown, currently mobile

On Oct 22, 2019, at 11:52, Nikhil Benesch ***@***.***> wrote:  What platform do you use, @Roguelazer? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[benesch]

I came up with a solution here that I think will satisfy everyone. Bindings are again checked into version control, so bindgen/libclang/clang are no longer build time dependencies. This time, however, the bindings are not platform-specific (they depend on platform-specific types like libc::FILE as necessary), so they don't need to be regenerated once per platform like before.