Unable to run rdkafka with Alpine (node:6 and node:8)

[IdanAdar]

Also posted in the Alpine repository: gliderlabs/docker-alpine#385

We are using the following as a base image:

FROM node:8-alpine

RUN apk --no-cache add \
      bash \
      g++ \
      ca-certificates \
      lz4-dev \
      musl-dev \
      cyrus-sasl-dev \
      openssl-dev \
      make \
      python

We then use this base image in the following Dockerfile:

FROM the-base-image

# Create app directory
RUN mkdir -p /usr/src/app
RUN mkdir temp

# Copy package.json first to check if an npm install is needed
COPY .npmrc /temp
COPY package.json /temp
RUN cd temp && npm install --production
RUN cp -a /temp/node_modules /usr/src/app

# Bundle app source
COPY . /usr/src/app

WORKDIR /usr/src/app

ENV PORT 8080
EXPOSE 8080

RUN chmod +x environment.sh
CMD ["/bin/sh", "/usr/src/app/environment.sh"]

When the built image is then attempting to start, we get the following error in the log:

"message":"kafkaProducerConfig is: '{\"security.protocol\":\"sasl_ssl\",\"sasl.mechanisms\":\"PLAIN\",\"api.version.request\":true,\"broker.version.fallback\":\"0.10.2.1\",\"dr_cb\":false,\"message.send.max.retries\":2,\"retry.backoff.ms\":100,\"socket.keepalive.enable\":true,\"metadata.broker.list\":\"kafka01-prod02.****.****.****.****.net:9093,kafka02-prod02.****.****.****.****.net:9093,kafka03-prod02.****.****.****.****.net:9093,kafka04-prod02.****.****.****.****.net:9093,kafka05-prod02.****.****.****.****.net:9093\",\"sasl.username\":\"*****\",\"sasl.password\":\"****\",\"ssl.ca.location\":\"/etc/ssl/certs\"}'\nkafkaTopicConfig is: '{\"request.required.acks\":0,\"request.timeout.ms\":5000,\"message.timeout.ms\":300000}'\nproducerOpts is: '{\"pollInterval\":100,\"DEFAULTS\":{\"POLL_INTERVAL\":100}}'","instanceId":"SystemId","module":"metadata-producer [Producer]","execTime_date":"2018-02-25T09:25:39.613Z","logLevel":"DEBUG"}

{"message":"Init error: '\"Couldn't connect to Kafka. Reason is: Error: Failed to create thread: No such file or directory (2). .\"'","instanceId":"SystemId","module":"metadata-producer","execTime_date":"2018-02-25T09:25:39.629Z","logLevel":"INFO"}

Has anyone encountered this issue with rdkafka and Alpine and knows of a workaround or solution?

[webmakersteve]

alpine has a lightweight version of glibc that causes a lot of problems with threading. I believe you need a full featured glibc in order to get it to work.

I am going to try to make an alpine base image since it is so commonly requested that solves these issues.

[IdanAdar]

Thanks @webmakersteve, I hope that you'll be our Kafka savior. :)

[webmakersteve]

Can you try installing from master and tell me if it works? I made changes to the way the library builds librdkafka that may fix all the alpine problems.

[webmakersteve]

https://github.com/Blizzard/node-rdkafka/blob/master/examples/docker-alpine.md

I was finally able to get it run with the new build like the documentation suggests! Hopefully this is the end of the alpine problems ;)

[IdanAdar]

@webmakersteve Unfortunately I still get the following when using the above as a base image.

Failed to create thread: No such file or directory (2).

Tested with v2.0.0 and 2.3.0.
BTW, the build log shows lots of warnings and deprecation notices: build.log

[webmakersteve]

Sorry - can you try 2.3.1? I was able to successfully run this code using the alpine image listed above:

const kafka = require('node-rdkafka');

var p = new kafka.Producer({ 'bootstrap.servers': 'localhost:9092' }, {});

p.connect({ timeout: 1000 }, function(err) {
  if (!err) {
    p.disconnect();
  } else {
    process.exit(0);
  }
});

[IdanAdar]

Now works with the example Dockerfile and v2.3.2.
Thanks @webmakersteve!