Can a new Consumer start from a specified offset?

[kchristidis]

Is there an API in this library that allows this?

I'm looking for something that basically taps into sarama's ConsumePartition() API and its offset argument.

Thank you.

[kchristidis]

So far, this is the only way I can make it work: Wait for a subscription to the topic/partition to be established, and once the partition consumer is launched, I MarkPartitionOffset() with offset-1 and then CommitOffsets(). Is the norm?

And if it is, assume I now want to start from sarama.OffsetOldest or sarama.OffsetNewest - how would I go about doing that? (Besides creating a new consumer in an entirely different consumer group.)

[dim]

Hey Kostas, there is no API for this in sarama-cluster, which is really all about coordinating consumption of a continuous topic. For lower-level tasks, please take a look at sarama itself, where you can manipulate partition offsets for a consumer group. In case you want to reset them, just commit the offset for each partition with a really short RetentionTime (e.g. 0 or 1, it's in seconds), which means it will be wiped after that time and you can start with a clean sheet. Hope that helps!

[kchristidis]

Gotcha. Thanks for the pointer!