Kafka container prematurely exits about half of the time

[ehashman]

We saw an example of this in our latest Travis build on my Docker PR (here), and I've also encountered this in my dev environment. We should track down what the problem is and fix it if possible.

Current workaround is to run docker-compose up again until the Kafka container doesn't die.

[ehashman]

Logs say:

[2016-02-18 18:56:18,519] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.

so @sirsean's inclination that this was a Zookeeper issue sounds about right to me.

[lvh]

I seem to remember someone mentioning that they had found more details on this somewhere. If that's the case, please update this ticket.

[ehashman]

I suspect that this bug has something to do with Kafka's interactions with Zookeeper. From the kafka-users mailing list: http://mail-archives.apache.org/mod_mbox/kafka-users/201305.mbox/%3CFF142F6B499AE34CAED4D263F6CA32901CAA9F88@EXTXMB19.nam.nsroot.net%3E

I also see an instance of this error related to non-graceful shutdown that a PR exists for upstream on the container image that we're using, but that doesn't seem to describe the clean creation case: wurstmeister/kafka-docker#58

[ehashman]

Closed by #85.