Add the ability to rewind offsets based on time

[aravindvs]

Recent Kafka and sarama has the ability to fetch offsets by timestamp;
see:
https://github.com/Shopify/sarama/wiki/Frequently-Asked-Questions

This is very helpful in cases where we need to rewind the offset and
reset our consumption to some earlier time. Some example is, let's say
the downstream app goes down or something and we need to rewind to some
time earlier.

This patch will help achieve this by rewinding the offset for the given
topic for all the paritions to which this consumer worker subscribes to.
This should make sure we "pause" the current consumption, reset the
offset and then resume (rebalance) the consumption before returning.

Also added a UT to rewind the offset and an example test on how to use
the API

[aravindvs]

@dim - please take a look and see if this or something equivalent to this can be done.. this will be very useful especially now with the ability to get offsets from timestamp supported in both Kafka and sarama lib..

[dim]

@aravindvs will try, probably later this week, thanks for this!

[prune998]

+1, we definitly need this !

[jerrygb]

+1, it would helpful for us as well. 👍

I not sure about convention used by the project to vote on the commit. Sorry if its incorrect :)

[dim]

hey all, apologies again for the delay, I will take a closer look in the next few days.

[gvishnutej]

@dim Would love to use this rewind offset feature, hope this PR will be merged soon

[prune998]

too bad nobody is taking care...

[dim]

I'm really sorry, but I'm travelling at the moment. I will review the PR as soon as I can.

…

On 23 Feb 2018 06:41, "Prune Sebastien THOMAS" ***@***.***> wrote: too bad nobody is taking care... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#179 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABA5huwSvK3iyRhsw0eVhdbewHpEWeKks5tXqPpgaJpZM4QQP9Q> .

[prune998]

thanks @dim, have a good trip :)

[davecb]

I'd like to add myself to the list of consumers of this improvement: I have a log extractor that awaits it.

[dim]

Hey all, I am currently working on the integration of a low level consumer group into sarama, see here: IBM/sarama#1083. It allows users to write their own, custom consumer implementations. Once accepted, I will work on the higher level cluster consumer (sarama-cluster V3) where I will aim to provide this feature.