Subscribe.AutoPartitionWithManualOffset closes over actor instance

[janm399]

When sending Subscribe.AutoPartitionWithManualOffset to the KafkaConsumerActor, the assignedListener and revokedListener functions may end up capturing the actor instance. The current API also makes it difficult to use non-blocking code to load the offsets.

It would be more convenient to have

final case class AutoPartitionWithManualOffset(
  topics: Iterable[String],
  assignedListener: List[TopicPartition] => Future[Offsets],
  revokedListener: List[TopicPartition] => Future[Unit]
) extends Subscribe

For safe use in actors, it would be useful to have a wrapper around assignedListener and revokedListener that works by asking a supplied ActorRef for a response (Offsets) to AssignedListeners(topics: List[TopicPartition) and RevokedListeners(...).

[jpallari]

Yeah, that would be nice indeed. The problem is that the underlying Java client doesn't support asynchronous listeners. The Java client revocation/assignment hooks must complete their work during the call.

[janm399]

I tried it at https://github.com/janm399/scala-kafka-client/tree/fmp; as far as I can tell, the main point of integration is consumer.seek(...), which can happen in onComplete.