Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
KAFKA-7609: Add Protocol Generator for Kafka #5893
clients/src/main/java/org/apache/kafka/common/protocol/Readable.java, Writable.java, ByteBufferAccessor.java
core/src, connect/src, clients/src/main/java/org/apache/kafka/clients/admin, etc.
left a comment
This looks good to me, Colin. I left a couple minor comments.
As a more general comment, I think we should include the docstrings in the schema files, so that we can eventually generate the protocol docs off of them. And if we're going to do that, we may actually want to be more precise about not changing field names from the current manual schemas, so that people don't get thrown off by changed names. I don't think either of these things necessarily has to happen in this patch, though.
2 times, most recently
Nov 21, 2018
That's a good point. I added documentation to all JSON schema files, based on the original Struct documentation. In cases where this documentation was missing, I created protocol documentation for the fields. I also ported over the comments explaining the purpose of each protocol version change.
I think it's best to decide about field names when doing the conversion for each RPC.
I fixed this so that we handle duplicates the same way we do now: by setting an INVALID_REQUEST error for them. This is actually specified in the KIP that added CreateTopics.
In order to make this possible, deserialization needs to be able to represent duplicate topics in the set. I handled this by using multisets instead of sets for data received over the wire. I think this is the correct approach in general because it's flexible and will allow us to covert over the existing code more easily. Throwing a deserialization error when encountering duplicates would be very unfriendly-- deserialization errors just result in the broker disconnecting without an error message.
To make this easier to review, I have split the CreateTopics changes off into a separate PR, #5972 . I think with that stuff split out, this is a very low-impact change since it's just adding a code generator, and not modifying any existing broker code path.