Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any plans to expose offsetsForTimes method on Consumer #228

Closed
denisivan0v opened this issue Jun 16, 2017 · 10 comments
Closed

Any plans to expose offsetsForTimes method on Consumer #228

denisivan0v opened this issue Jun 16, 2017 · 10 comments

Comments

@denisivan0v
Copy link
Contributor

Java consumer API has the method

Map<TopicPartition,OffsetAndTimestamp> offsetsForTimes(Map<TopicPartition,Long> timestampsToSearch)

which helps to assign a consumer to a specified offset based on timestamp.

Is there any plans to support this in .NET API?

@mhowlett
Copy link
Contributor

Should be easy to add (corresponding function exists in librdkafka) but hasn't been prioritized - you're the first person to ask for the dotnet client I think.

PRs accepted :-)

@denisivan0v
Copy link
Contributor Author

I'll take a look. Will be appreciated for docs/guidelines describing how to work with librdkafka correctly.

@denisivan0v
Copy link
Contributor Author

denisivan0v commented Jun 19, 2017

Here is the corresponding Kafka improvement proposal: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65868090. And the librdkafka issue: confluentinc/librdkafka#842

Also, found the similar issue #85

@mhowlett
Copy link
Contributor

The relevant librdkafka function is rd_kafka_offsets_for_times.
This should be wrapped in LibRdKafka.cs (for examples, see other methods).
Then exposed as a method on Consumer.

Ahh yes, I forgot someone else asked for this :-). It'd be a quick win, we should do this. Any help appreciated, but I'll try to get to it sooner rather than later.

@denisivan0v
Copy link
Contributor Author

denisivan0v commented Jun 19, 2017

I've started to implement this method, but stuck at passing arguments to librdkafka.
Here is my commit denisivan0v@8ff96ae.

Now I need to convert IEnumerable<KeyValuePair<TopicPartition, long>> instance to rd_kafka_topic_partition_list_t instance to call the corresponding method https://github.com/edenhill/librdkafka/blob/master/src/rdkafka.c#L2433. But I'm really not sure how to do it.

@mhowlett, can you give me an advice on this? Comments on current changes are also appreciated.

@mhowlett
Copy link
Contributor

Have a look in SafeKafkaHandle for something similar... perhaps GetCTopicPartitionList. This conversion should happen in SafeKafkaHandle - sorry didn't think of that before.

@denisivan0v
Copy link
Contributor Author

denisivan0v added a commit to denisivan0v/confluent-kafka-dotnet that referenced this issue Jun 20, 2017
mhowlett pushed a commit that referenced this issue Jul 25, 2017
* WIP: Implementing Consumer.OffsetsForTimes. Interacting with librdkafka

* #228 Consumer.OffsetsForTimes implemented

* Changes on review comments (#235)

* Changes for timeout param on Consumer.OffsetsForTimes API (#235)

* Bugfix: null checks in Producer ctor

* Intergration test for Consumer.OffsetsForTimes added (#235)

* Unit tests added for TopicPartitionTimestamp (#235)

* changes on codereview (#235)

* argument renaming in rd_kafka_offsets_for_times (#235)

* fixed incorrect merge

* removed space

* changes on codereview (#235)

* small code cleaning in tests

* minor changes
@melnikov77
Copy link

Any plans to release this method? I can't find it in "0.11.0.x" branch :-(

@mhowlett
Copy link
Contributor

it will be in the next major release, or you can use a package from the appveyor nuget feed.

@jake0029
Copy link

Sorry, do we have sample usage of OffsetFortimes event in .net code? Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants