Error compiling librdkafka

[lephyrius]

I tried using this gem but I got this error message when compiling:

Downloading v0.11.5.tar.gz ( 98%) 
Downloading v0.11.5.tar.gz (100%)
Extracting v0.11.5.tar.gz into tmp/x86_64-apple-darwin17.7.0/ports/librdkafka/0.11.5... OK
Running 'configure' for librdkafka 0.11.5... OK
Running 'compile' for librdkafka 0.11.5... ERROR, review '/Users/thedude/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/bundler/gems/rdkafka-ruby-86ee6932b2e2/ext/tmp/x86_64-apple-darwin17.7.0/ports/librdkafka/0.11.5/compile.log' to see
what happened. Last lines are:
========================================================================
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_SHA512", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_X509_STORE_set_flags", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_X509_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_X509_new", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_X509_verify_cert_error_string", referenced from:
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_d2i_PKCS12_fp", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_sk_pop_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [librdkafka.1.dylib] Error 1
make: *** [libs] Error 2
========================================================================
rake aborted!
Failed to complete compile task

Tasks: TOP => default
(See full trace by running task with --trace)

rake failed, exit code 1

Gem files will remain installed in /Users/thedude/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/bundler/gems/rdkafka-ruby-86ee6932b2e2 for inspection.
Results logged to /Users/thedude/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/bundler/gems/extensions/x86_64-darwin-17/2.5.0-static/rdkafka-ruby-86ee6932b2e2/gem_make.out

Using latest mac os and xcode. Also I got the same message on the release and latest master.

[thijsc]

Looks like the compile process is having trouble locating OpenSSL. Are you using OpenSSL 1.0? How did you install it?

We still need to do some work on installation instructions.

[lephyrius]

@thijsc I installed it using Homebrew.
It is this version:

openssl: stable 1.0.2o (bottled) [keg-only]
SSL/TLS cryptography library
https://openssl.org/
/usr/local/Cellar/openssl/1.0.2o_1 (1,791 files, 12.3MB)
  Poured from bottle on 2018-03-30 at 07:54:14
/usr/local/Cellar/openssl/1.0.2o_2 (1,792 files, 12.3MB)
  Poured from bottle on 2018-06-25 at 11:21:24

[thijsc]

Are any of the tips here helpful?
confluentinc/librdkafka#1639

It’s not really in the scope of this gem to handle this. But i would like to add some error handling with debug output and hints on what to do. If you can do a bit of digging that would be very good input.

[thijsc]

@lephyrius did you make progress on this?

[thijsc]

Closing until more information is added here.

[adamof]

In case anyone else is seeing this on os X, I was able to install the library by specifying the following paths to openssl libs:

CPPFLAGS="-I/usr/local/opt/openssl/include" LDFLAGS="-L/usr/local/opt/openssl/lib" gem install rdkafka

The values are taken from the brew post-installation message of openssl (brew install openssl)

[thijsc]

Thanks for that! I'm considering adding some platform detection logic to set sensible defaults like this.

[thijsc]

@lephyrius I might have fixed this in #61. Could you test by https://github.com/appsignal/rdkafka-ruby#development in that branch?

[thijsc]

Closing this for now, hopefully #61 improves things. If not we'll reopen.

[jvanbaarsen]

@thijsc I had the same problem today. The fix from @adamof solved it for me.
-- edit: I just realize that the fix you mentioned above is not yet deployed, so that was the reason I had to use the work-around.