Consumer groups on Kafka 0.9

[dim]

Hey, I am working on implementing consumer groups on Kafka 0.9 and here are the required API additions. I am not particularly sure about metadata, as Kafka doesn't prescribe but only recommend a structure for it. I followed the recommendations on https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-JoinGroupRequest, please let me know what you think.

Cheers,
Dim

[wvanbergen]

We could check this in Config.Validate()

[wvanbergen]

Thanks for working on this 👍

As for the approach, I think what you are doing makes sense:

• We should follow the protocol's recommended structure for consumer groups.
• Some of the protocol identifiers have been renamed. Due to backwards compatibility reasons, we cannot really do these renames, unless we declare a 2.0 version (something that we are considering). For now, it's probably best to just build it with the current names; the functionality is not really different.
• Let's start with these building blocks, and tackle the higher level consumer construct in a separate PR when we land this.

[dim]

@wvanbergen so yes, I am working on a high-level group consumer as a separate package for now. I have addressed your feedback above. Please let me know if there's anything else.

[wvanbergen]

As for naming, we should probably call this ConsumerGroupMemberMetadata. This is a specialization of the generic group type.

[wvanbergen]

This looks pretty good to me. Let's see what @eapache thinks.

[dim]

Hey, I have released a v2 branch of https://github.com/bsm/sarama-cluster/tree/v2 which requires this PR to be merged. I would really appreciate if you could do it next week. Feedback on the cluster consumer is also welcome. Thanks, dim

[eapache]

I've been away, sorry. I'll take a look tomorrow.