Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consumers in consumer group gets no partitions assigned to them. #1363

Open
dev-websplanet opened this issue Dec 8, 2019 · 1 comment
Open

Comments

@dev-websplanet
Copy link

dev-websplanet commented Dec 8, 2019

Environment
Node version: 10.15.3
Kafka-node version: 5.0.0
Kafka version (single node): 2.3.0
Number of Brokers: 1
Number of partitions for topic: 1

my app consists on micro services, and each micro service can have 1 - 3 instances running (only one should receive each message).
so if i understood the correct usage of kafka consumer groups, when each ms is started it creates a consumer group with the groupId of the ms name (unique per ms), each topic is added with 1 partition, and i figured only one consumer will consume the messages from the only partition, and if the consumer leaves the group some other consumer will start consuming from that partition.

sometimes the consumer group is created and the topics are added but i can see when i run commands from the kafka docker that no consumer is assigned to the partition, and i cant find the reason.

command: kafka-consumer-groups.sh --describe --group ums --bootstrap-server localhost:9092

valid response:

TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID

resConfig_update 0 0 0 0 ums-consumer-1575794029875-4f78bb47-84f9-4d75-bede-3b838802723f /172.16.103.20 ums-consumer-1575794029875

reseller_delete 0 0 0 0 ums-consumer-1575794029875-4f78bb47-84f9-4d75-bede-3b838802723f /172.16.103.20 ums-consumer-1575794029875

reseller_create 0 0 0 0 ums-consumer-1575794029875-4f78bb47-84f9-4d75-bede-3b838802723f /172.16.103.20 ums-consumer-1575794029875

sometimes i get this:

TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID -

resConfig_update 0 0 0 0 - - -

reseller_delete 0 0 0 0 - - -

reseller_create 0 0 0 0 - - -

im seding a SyncGroupRequest from the consumer to debug the the problem and i can see the consumer has no partitions assigned to him.
this is the consumerGroup creation:

`
let options = {
id: "ums-consumer-${Date.now()}",
clientId: "ums-consumer-${Date.now()}",
kafkaHost: host,
groupId: 'ums',
protocol: ['roundrobin'],
idleConnection: 60000,
fromOffset: 'earliest'};

let ConsumerGroup = kafka.ConsumerGroup;
let consumer = new ConsumerGroup(options, ['reseller_create', 'reseller_delete', 'resConfig_update']);
`

i saw a post about a similar problem but the solutions did not work, and the one who post it just changed clients eventually.
post: #339

@0xslipk
Copy link

0xslipk commented Feb 3, 2020

@dev-websplanet maybe is related to this issue #1355

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants