Question about consumer group member failover #273
Description
Environment:
- cppkafka: master branch
- librdkafka: v1.6.1
- Kafka: 2.7.0
I create 2 consumers (c01, c02) by using /examples/consumer_example.cpp to form a consumer group (g01) to fetch data from a Kafka topic (t01) with single partition. I expect that one of consumer group members fetches data, the rest member is in stand-by for failover.
2 consumers are created as follows
c01:
$ ./consumer_example -b kafka01l:9092,kafka02:9092,kafka03:9092 -t t01 -g g01
c02:
$ ./consumer_example -b kafka01l:9092,kafka02:9092,kafka03:9092 -t t01 -g g01
When c01 and c02 are running, one (e.g., c01) gets assigned to the partition of topic t01. I then manually stop c01, consumer group goes into rebalancing but finds no active member finally.
By enabling debug option { "debug", "cgrp" } in consumer_example.cpp, c02 shows the following when c01 is stopped.
%7|1615262787.137|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/2: Heartbeat for group "g01" generation id 190
%7|1615262787.138|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "g01" heartbeat error response in state up (join-state wait-assign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1615262790.137|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/2: Heartbeat for group "g01" generation id 190
%7|1615262790.138|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "g01" heartbeat error response in state up (join-state wait-assign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1615262793.137|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/2: Heartbeat for group "g01" generation id 190
%7|1615262793.138|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "g01" heartbeat error response in state up (join-state wait-assign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
With the same Kafka broker settings, I try to create 2 consumers in a consumer group by using consumer example in librdkafka v1.6.1 to consume data in the partition of topic t01, member failover works fine.
Do you have any advice for my consumer group member failover issue? Thanks.