Skip to content

KAFKA-19466: LogConcurrencyTest should close the log when the test completes; migrate LogConcurrencyTest #20110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: trunk
Choose a base branch
from

Conversation

Yunyung
Copy link
Collaborator

@Yunyung Yunyung commented Jul 6, 2025

  • Fix testUncommittedDataNotConsumedFrequentSegmentRolls() and
    testUncommittedDataNotConsumed(), which call createLog() but never close
    the log when the tests complete.
  • Move LogConcurrencyTest to the Storage module and rewrite it in Java.

Jira: https://issues.apache.org/jira/browse/KAFKA-19466 Reviewers:
Chia-Ping Tsai chia7712@gmail.com

@github-actions github-actions bot added triage PRs from the community core Kafka Broker tests Test fixes (including flaky tests) storage Pull requests that target the storage module labels Jul 6, 2025
Copy link
Member

@chia7712 chia7712 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yunyung thanks for this migration


validateConsumedData(log, consumer.getConsumedBatches());
} finally {
executor.shutdownNow();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we wait for all threads to terminate?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Thanks.

}

@Override
public Void call() throws Exception {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the method getConsumedBatches could be removed if call returns List<FetchedBatch>

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

@github-actions github-actions bot removed the triage PRs from the community label Jul 7, 2025
Copy link
Member

@chia7712 chia7712 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yunyung thanks for this patch


@Test
public void testUncommittedDataNotConsumed() throws Exception {
testUncommittedDataNotConsumed(createLog());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the log get closed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, log is closed in the teardown method.

);
}

private record FetchedBatch(long baseOffset, int epoch) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it could be replaced by LinkedHashMap?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Kafka Broker storage Pull requests that target the storage module tests Test fixes (including flaky tests)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants