Refactor Consumer Group to use Client

[rhansen2]

No description provided.

[achille-roussel]

We discussed the PR during our team meeting today and agreed on making the following changes:

• base the change on the 0.4 branch since it contains minor modifications to the exported APIs (removed an unused field + change fields from int32 to int types)
• explore passing a parent context to the methods of the coordinator interface
• add the AllowAutoTopicCreation field to the top level MetadataRequest type so we don't need to bypass the abstraction layers

[achille-roussel]

This is looking ready to merge in my opinion 🙌

[moolitayer]

Hi @rhansen2 @achille-roussel,

Thank you for working on this awsome library!

Recently we've seen an issue on upgrading from v0.4.35 to v0.4.36. The issue went away after downgrad:

Failed to join X: [79] Member ID Required: the group member needs to have a valid member id before actually entering a consumer group
Failed to join group X: kafka.(*Client).JoinGroup: unexpected EOF
Failed to sync group X: kafka.(*Client).SyncGroup: unexpected EOF
Failed to sync group X: [27] Rebalance In Progress: the coordinator has begun rebalancing the group, the client should rejoin the group
kafka.(*Client).SyncGroup: unexpected EOF

Do you think it might be ralted to this change? Should I create an issue for it with our env details + code?

[rhansen2]

Could be related. Please open an issue and we'll take a look, thanks!

[jhartzell]

Hi @rhansen2 @achille-roussel,

Thank you for working on this awsome library!

Recently we've seen an issue on upgrading from v0.4.35 to v0.4.36. The issue went away after downgrad:

Failed to join X: [79] Member ID Required: the group member needs to have a valid member id before actually entering a consumer group
Failed to join group X: kafka.(*Client).JoinGroup: unexpected EOF
Failed to sync group X: kafka.(*Client).SyncGroup: unexpected EOF
Failed to sync group X: [27] Rebalance In Progress: the coordinator has begun rebalancing the group, the client should rejoin the group
kafka.(*Client).SyncGroup: unexpected EOF

Do you think it might be ralted to this change? Should I create an issue for it with our env details + code?

Also seeing the same issue under a Strimzi cluster

[rhansen2]

@moolitayer or @jhartzell Are either of you able to create an issue with a reproduction of the issues you're seeing?

[rhansen2]

@moolitayer @jhartzell
While not exactly what you're seeing, I did find a potential source of unexpected EOFs. Would you be able to try out https://github.com/rhansen2/kafka-go/tree/join-group-meta-parsing and see if that changes anything for you?

Thanks!