Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSK - KafkaConsumer failing to connect silently #20

Open
wbarnha opened this issue Mar 8, 2024 · 0 comments
Open

MSK - KafkaConsumer failing to connect silently #20

wbarnha opened this issue Mar 8, 2024 · 0 comments

Comments

@wbarnha
Copy link
Owner

wbarnha commented Mar 8, 2024

I think I found an issue connecting to MSK.

In the Kafka-python documentation, it states that if at least one host of the connection is up and able to respond to the Metadata API Request, the connection should be fine.

However, connection is failing silently; I only noticed because messages were not being consumed.

Does that relate to dpkp#2390 and dpkp#2281?

Kafka-python 2.0.2, consumer is running on a docker image hosted in an AWS EC2.
My configuration is straightforward:
consumer = KafkaConsumer( 'mytopic', group_id=group_id, bootstrap_servers=['host1:port1','host2:port2','host3:port3','host4:port4'], security_protocol='SSL' )
This works when all hosts are healthy but fails if there is a typo on the hostname or a host is unhealthy.

For this bug, I would expect that connection to succeed despite one host not being accessible as long at least of the list it is. And in the worst case scenario a failure to connect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant