New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KAFKA-14462; [14/N] Add PartitionWriter #13675
Conversation
core/src/main/scala/kafka/coordinator/group/PartitionWriterImpl.scala
Outdated
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/PartitionWriterImpl.scala
Outdated
Show resolved
Hide resolved
core/src/test/scala/unit/kafka/coordinator/group/PartitionWriterImplTest.scala
Outdated
Show resolved
Hide resolved
core/src/test/scala/unit/kafka/coordinator/group/PartitionWriterImplTest.scala
Outdated
Show resolved
Hide resolved
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/RecordSerializer.java
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/PartitionWriterImpl.scala
Outdated
Show resolved
Hide resolved
Hey David -- whenever we update appendRecords we have to modify a ton of tests that mock the method. (Basically add a ton of any(), since the mocker doesn't get default args). There are 283 failing now 😅 See here for an example of a PR that updates them: https://github.com/apache/kafka/pull/13391/files |
Yeah, I have noticed all the failed tests. I will fix them. |
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Outdated
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Outdated
Show resolved
Hide resolved
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java
Show resolved
Hide resolved
These connect and kraft tests are a little concerning. They seem unrelated but maybe we should file a jira. |
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Outdated
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Show resolved
Hide resolved
core/src/test/scala/unit/kafka/coordinator/group/CoordinatorPartitionWriterTest.scala
Show resolved
Hide resolved
core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala
Show resolved
Hide resolved
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/RecordSerializer.java
Show resolved
Hide resolved
group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordSerializerTest.java
Show resolved
Hide resolved
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/PartitionWriter.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
This patch introduces the
PartitionWriter
interface in thegroup-coordinator
module. TheReplicaManager
resides in thecore
module and it is thus not accessible from thegroup-coordinator
one. ThePartitionWriterImpl
is basically an implementation of the interface residing incore
which interfaces with theReplicaManager
.One notable difference from the usual produce path is that the
PartitionWriter
returns the offset following the written records. This is then used by the coordinator runtime (coming later) to track when the request associated with the write can be completed.Committer Checklist (excluded from commit message)