Feature request: Connection error handling

[Paxa]

Currently racecar will not call error handler when can not connect to kafka (dns error, host not reachable, open connection error, etc...)

Temporary monkey patch:

Racecar::Runner.class_eval do
  alias_method :run_origin, :run

  def run
    run_origin
  rescue Kafka::ConnectionError => error
    config.error_handler.call(error)
    raise error
  end
end

[dasch]

Do you have a monitoring solution set up in addition to your exception tracker? Connection errors are not exceptional in Kafka, so typically you want to use metrics based alerts rather than exception trackers.

[Paxa]

Do you have a monitoring solution set up in addition to your exception tracker?

Not yet, still thinking how to make it right. The only option I can think of is detecting if process/container exit with error code too often

Connection errors are not exceptional in Kafka

I got some kafka related errors too, e.g. Kafka::NoPartitionsAssignedError, it can connect to port on remote server, but can not start receiving data

[mensfeld]

@Paxa the no partition error occurs when you have more consumers than partitions in a topic.

[dasch]

I would recommend using e.g. Datadog or Librato with Statsd to track metrics. Connection errors is one metric published by ruby-kafka.

[Paxa]

@dasch Thank you!