Allows a maximum of 1 connection to zookeeper per (pod) IP #230
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
one zk connection per pod should be sufficient in Kubernetes. I'm adding this config because it's represented in https://github.com/apache/kafka/blob/2.1/config/zookeeper.properties
|
I'd like to keep this limit as "for microservices". There really shoudn't be more than one zk client per IP, unless maybe multiple containers in the same pod needs zk access. Such a requirement seems unlikely though, as Kafka's client libs have been undergoing a transition towards not contacting zk directly. Access to zookeeper from outside the cluster could be another use case, but as some threads in this project have discussed I don't think it's feasible. |
solsson
added a commit
that referenced
this pull request
Jun 19, 2019
for example kafka-topics --create --if-not-exists. This commit constitues an instance of "anyone sees practical issues" in #230 :)
solsson
added a commit
that referenced
this pull request
Jun 25, 2019
for example kafka-topics --create --if-not-exists. This commit constitues an instance of "anyone sees practical issues" in #230 :)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of the 5.0 destabilization effort :)
I like this kind of restrictions as sanity check. It discourages use of the Kafka pods for administration stuff (bad practice wrt resource limits etc) and use of legacy clients that connect directly to zookeeper. Zookeeper docs also mentions "file descriptor exhaustion".
See the commit comment for the rationale behind the number 1. Sure, we'll increase it if anyone sees practical issues. For now it's on par with the philosophy behind memory limits in 5.0.0.