Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION] I am using AIOkafka, but i have some question. #933

Closed
taeyoung-lee-tpv-clt opened this issue Oct 24, 2023 · 1 comment
Closed
Labels

Comments

@taeyoung-lee-tpv-clt
Copy link

while I'm using AIOkafka but sometimes when I using 'consumer.commit()'
At that time, I found this error on the log
'commit cannot be completed since the group has already rebalanced'

so when i find some information.
then they said to me the AIOkafka have an autocommoit.
so I changed to passive-commit = False

I use this code 'consumer.commit()'.
I've written it the end of the process after service part

service1.handler()
service2.handler()
service3.handler()

consumer.commit()
like this

so but how can i overcome this horrible situation
i want to solve this problem of rebalanced error when i commit()

eventually, i changed poll size and ms time

so please answer to me except for this option

thank you

@vmaurin
Copy link
Contributor

vmaurin commented Oct 24, 2023

Maybe my question could help you #929

A main difference between the java kafka client and aiokafka is that the group rebalancing is managed in a background coroutine, so the partition assignment could change while you are processing records and before you commit them.

For me it feels like in real production environment, with multiple consumer, the commit() method without argument should be avoided

@aio-libs aio-libs locked and limited conversation to collaborators Oct 25, 2023
@ods ods converted this issue into discussion #934 Oct 25, 2023

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
Projects
None yet
Development

No branches or pull requests

2 participants