Skip to content
This repository has been archived by the owner on Mar 24, 2021. It is now read-only.

Commit

Permalink
add note on connection loss handling to usage guide
Browse files Browse the repository at this point in the history
  • Loading branch information
emmettbutler committed May 7, 2018
1 parent fd8e893 commit e7dd309
Showing 1 changed file with 27 additions and 0 deletions.
27 changes: 27 additions & 0 deletions doc/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,30 @@ memory instead of letting them be garbage collected and reinstantiated repeatedl
topic_producers = {topic.name: topic.get_producer() for topic in topics_to_produce_to}
for destination_topic, message in consumed_messages:
topic_producers[destination_topic.name].produce(message)


Handling connection loss
========================

The pykafka components are designed to raise exceptions when sufficient connection to
the Kafka cluster cannot be established. There are cases in which some but not all of
the brokers in a cluster are accessible to pykafka. In these cases, the component will
attempt to continue operating. When it can't, an exception will be raised. Often this
exception will be either `NoBrokersAvailableError` or `SocketDisconnectedError`. These
exceptions should be caught and the component instance should be reinstantiated. In some
cases, calling `stop(); start()` in response to these exceptions can be enough to
establish a working connection.

.. sourcecode:: python

from pykafka.exceptions import SocketDisconnectedError, NoBrokersAvailableError
# this illustrates consumer error catching; a similar method can be used for producers
consumer = topic.get_simple_consumer()
try:
consumer.consume()
except (SocketDisconnectedError, NoBrokersAvailableError) as e:
consumer = topic.get_simple_consumer()
# use either the above method or the following:
consumer.stop()
consumer.start()

0 comments on commit e7dd309

Please sign in to comment.