Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KAFKA-14568: Move FetchDataInfo and related to storage module #13085

Merged
merged 9 commits into from Jan 13, 2023
Merged

KAFKA-14568: Move FetchDataInfo and related to storage module #13085

merged 9 commits into from Jan 13, 2023

Conversation

fvaleri
Copy link
Collaborator

@fvaleri fvaleri commented Jan 6, 2023

Part of KAFKA-14470: Move log layer to storage module.

@fvaleri
KAFKA-14568: Move FetchDataInfo and related to storage module
64fcaf5 
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
@fvaleri
Copy link
Collaborator Author

fvaleri commented Jan 6, 2023

@ijuma @satishd fyi

@fvaleri
Change FetchIsolation and related classes to enum
4139777 
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
@fvaleri
Comments from Ismael
2c90f33 
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
@fvaleri
Copy link
Collaborator Author

fvaleri commented Jan 7, 2023 •
edited

Let's wait for CI builds, but it should be good now. Thanks for the review.

ijuma
ijuma reviewed Jan 8, 2023
View reviewed changes
Copy link
Contributor

@ijuma ijuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a pass and left some comments. Note that the fetch path is performance sensitive, so we have to be careful with collection conversions and other such things.

clients/src/main/java/org/apache/kafka/common/utils/FetchRequestUtils.java Outdated Show resolved Hide resolved
core/src/main/scala/kafka/cluster/Partition.scala Show resolved Hide resolved
core/src/main/scala/kafka/log/LocalLog.scala Outdated Show resolved Hide resolved
core/src/main/scala/kafka/server/ReplicaManager.scala Outdated Show resolved Hide resolved
storage/src/main/java/org/apache/kafka/server/log/internals/FetchIsolation.java Outdated Show resolved Hide resolved
storage/src/main/java/org/apache/kafka/server/log/internals/FetchParams.java Outdated Show resolved Hide resolved
storage/src/main/java/org/apache/kafka/server/log/internals/FetchParams.java Outdated Show resolved Hide resolved
storage/src/main/java/org/apache/kafka/server/log/internals/FetchParams.java Show resolved Hide resolved
storage/src/main/java/org/apache/kafka/server/log/internals/FetchDataInfo.java Show resolved Hide resolved
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala Outdated Show resolved Hide resolved
@fvaleri
Copy link
Collaborator Author

fvaleri commented Jan 8, 2023

Note that the fetch path is performance sensitive, so we have to be careful with collection conversions and other such things.

Yes and thanks for the detailed reviews. Much appreciated.

@fvaleri
Additional comments from Ismael
591da01 
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
@fvaleri
Fix style errors
c42f986 
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
@fvaleri fvaleri requested a review from ijuma January 10, 2023 10:02
ijuma
ijuma approved these changes Jan 12, 2023
View reviewed changes
Copy link
Contributor

@ijuma ijuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a few clean-ups and it LGTM now. @fvaleri Are you good with them too?

@ijuma
Merge remote-tracking branch 'apache-github/trunk' into move-fdi
bc80173 
* apache-github/trunk:
  KAFKA-14601: Improve exception handling in KafkaEventQueue apache#13089
  KAFKA-14367; Add `OffsetCommit` to the new `GroupCoordinator` interface (apache#12886)
  KAFKA-14530: Check state updater more often (apache#13017)
  KAFKA-14304 Use boolean for ZK migrating brokers in RPC/record (apache#13103)
  KAFKA-14003 Kafka Streams JUnit4 to JUnit5 migration part 2 (apache#12301)
  KAFKA-14607: Move Scheduler/KafkaScheduler to server-common (apache#13092)
  KAFKA-14367; Add `OffsetFetch` to the new `GroupCoordinator` interface (apache#12870)
  KAFKA-14557; Lock metadata log dir (apache#13058)
  MINOR: Implement toString method for TopicAssignment and PartitionAssignment (apache#13101)
  KAFKA-12558: Do not prematurely mutate internal partition state in Mirror Maker 2 (apache#11818)
  KAFKA-14540: Fix DataOutputStreamWritable#writeByteBuffer (apache#13032)
  KAFKA-14600: Reduce flakiness in ProducerIdExpirationTest (apache#13087)
  KAFKA-14279: Add 3.3.x streams system tests (apache#13077)
  MINOR: bump streams quickstart pom versions and add to list in gradle.properties (apache#13064)
  MINOR: Update KRaft cluster upgrade documentation for 3.4 (apache#13063)
  KAFKA-14493: Introduce Zk to KRaft migration state machine STUBs in KRaft controller. (apache#12998)
  KAFKA-14570: Fix parenthesis in verifyFullFetchResponsePartitions output (apache#13072)
  MINOR: Remove public mutable fields from ProducerAppendInfo (apache#13091)
@ijuma
A couple of fixes
54304fa
@ijuma
Copy link
Contributor

ijuma commented Jan 13, 2023

JDK 8 and 11 builds passed, 17 had 2 unrelated failures.

@ijuma ijuma merged commit 111f02c into apache:trunk Jan 13, 2023
@fvaleri
Copy link
Collaborator Author

fvaleri commented Jan 13, 2023

I pushed a few clean-ups and it LGTM now. @fvaleri Are you good with them too?

Yes, good. Thanks.

@fvaleri fvaleri deleted the move-fdi branch January 13, 2023 08:34
ijuma added a commit to confluentinc/kafka that referenced this pull request Jan 17, 2023
@ijuma
Merge remote-tracking branch 'apache-github/trunk' into confluentinc/…
68d17cc 
…master

* apache-github/trunk: (23 commits)
  MINOR: Include the inner exception stack trace when re-throwing an exception (apache#12229)
  MINOR: Fix docs to state that sendfile implemented in `TransferableRecords` instead of `MessageSet` (apache#13109)
  Update ProducerConfig.java (apache#13115)
  KAFKA-14618; Fix off by one error in snapshot id (apache#13108)
  KAFKA-13709 (follow-up): Avoid mention of 'exactly-once delivery' or 'delivery guarantees' in Connect (apache#13106)
  KAFKA-14367; Add `TxnOffsetCommit` to the new `GroupCoordinator` interface (apache#12901)
  KAFKA-14568: Move FetchDataInfo and related to storage module (apache#13085)
  KAFKA-14612: Make sure to write a new topics ConfigRecords to metadata log iff the topic is created (apache#13104)
  KAFKA-14601: Improve exception handling in KafkaEventQueue apache#13089
  KAFKA-14367; Add `OffsetCommit` to the new `GroupCoordinator` interface (apache#12886)
  KAFKA-14530: Check state updater more often (apache#13017)
  KAFKA-14304 Use boolean for ZK migrating brokers in RPC/record (apache#13103)
  KAFKA-14003 Kafka Streams JUnit4 to JUnit5 migration part 2 (apache#12301)
  KAFKA-14607: Move Scheduler/KafkaScheduler to server-common (apache#13092)
  KAFKA-14367; Add `OffsetFetch` to the new `GroupCoordinator` interface (apache#12870)
  KAFKA-14557; Lock metadata log dir (apache#13058)
  MINOR: Implement toString method for TopicAssignment and PartitionAssignment (apache#13101)
  KAFKA-12558: Do not prematurely mutate internal partition state in Mirror Maker 2 (apache#11818)
  KAFKA-14540: Fix DataOutputStreamWritable#writeByteBuffer (apache#13032)
  KAFKA-14600: Reduce flakiness in ProducerIdExpirationTest (apache#13087)
  ...
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Jan 25, 2023
@fvaleri @ijuma @guozhangwang
KAFKA-14568: Move FetchDataInfo and related to storage module (apache…
eb30d6b 
…#13085)

Part of KAFKA-14470: Move log layer to storage module.

Reviewers: Ismael Juma <ismael@juma.me.uk>

Co-authored-by: Ismael Juma <ismael@juma.me.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ijuma ijuma ijuma approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@fvaleri @ijuma