How to handle offsetOutOfRange error

[nedstarksbastard]

I am running a node server that acts as a Kafka Consumer. I had seen instances of few offsetOutOfRange errors online, whereby the consumer tries to read messages that are no longer available. I tried to overcome that by having the following handler

consumer.on('offsetOutOfRange', function (topic) {
        console.log("------------- offsetOutOfRange ------------");
        topic.maxNum = 2;
        offset.fetch([topic], function (err, offsets) {
            var min = Math.min.apply(null, offsets[topic.topic][topic.partition]);
            consumer.setOffset(topic.topic, topic.partition, min);
        });
});

I am still not able to absolve this issue. I get the offsetOutOfRange issue in a continuous loop. But amazingly I am also getting all the messages generated out of kafka. Does anyone have a clue as to what is happening and how to solve it so that I dont get into this code block. I thought the consumer.setOffset(topic.topic, topic.partition, min); should be able to set the correct offset but that is not the case.

[Foo42]

We were experiencing the same loop and diagnosed the issue to be a type mismatch between the offset numbers returned from zookeeper and those in the payloads list (number in payloads, string from zookeeper). We were preparing a pull request and then noticed it had been fixed in 2747d7c