Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-14683 Migrate WorkerSinkTaskTest to Mockito (3/3) #15316

Merged
merged 29 commits into from Mar 6, 2024

Conversation

hgeraldino
Copy link
Contributor

Last batch for migrating WorkerSinkTaskTest tests to Mockito

@gharris1727 gharris1727 added connect tests Test fixes (including flaky tests) labels Feb 7, 2024
@divijvaidya
Copy link
Contributor

As part of this last batch, please also removed this test from the list here:

"**/WorkerSinkTaskTest.*"

@hgeraldino hgeraldino marked this pull request as ready for review February 13, 2024 01:27
@hgeraldino
Copy link
Contributor Author

Final batch of the WorkerSinkTaskTest migration 🎉 🎉 🎉

This one is probably the largest of the 3 (sorry).
We can either do one final commit renaming the test from WorkerSinkTaskTestMockito back to WorkerSinkTastTest, or just open a new PR once this gets merged to do the cleanup

Tagging @divijvaidya @gharris1727 @C0urante

Copy link
Contributor

@gharris1727 gharris1727 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hgeraldino, that last test is especially complex. Thank you for your attention to detail here!


rebalanceListener.getValue().onPartitionsRevoked(INITIAL_ASSIGNMENT);
rebalanceListener.getValue().onPartitionsAssigned(Collections.emptyList());
return new ConsumerRecords<>(Collections.singletonMap(TOPIC_PARTITION3, Collections.singletonList(newRecord)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the original test:

This confuses me. The earlier 3rd poll assigned topic-partition 3, and iteration 3 paused the newAssignment, including topic-partition 3. Yet in the 5th poll, this record is returned for a partition which should be paused.

The 5th iteration has the put succeed, and then the partitions are unpaused. But this unpause should only allow for new records to appear in the 6th poll, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, the test was a bit off mainly because the first call to task.put(..) shouldn't have thrown an exception.

Here's the sequence now (which matches the original WorkerSinkTaskTest):

  1. Iteration#1: partitions on INITIAL_ASSIGNMENT are assigned
  2. Iteration#2: task.put(...) throws, partitions are paused
  3. Iteration#3: P3 is assigned, task.put(...) throws (task is already paused so it's a noop)
  4. Iteration#4: task.put(...) throws, noop
  5. Iteration#5: initial assignment is revoked (pending messages for P1 are removed); consumer.poll(...) returns a record for P3, which is successfully processed.

I've added the corresponding verifications after each step. Hope it makes sense now

hgeraldino and others added 3 commits February 17, 2024 13:38
…/WorkerSinkTaskMockitoTest.java

Co-authored-by: Greg Harris <gharris1727@gmail.com>
@hgeraldino
Copy link
Contributor Author

Thanks @gharris1727 for your review. I think I addressed most of your comments.

If anyone has any other suggestions, please let me know. Ideally we should rename this file back to WorkerSinkTaskTest (not sure if as part of this PR or a separate one) and have checkstyle, build.gradle, etc. cleaned up

Copy link
Contributor

@gharris1727 gharris1727 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks @hgeraldino for all your hard work!

We can straighten out the build.gradle and move back to the old class name in a follow-up.

@gharris1727
Copy link
Contributor

Test failures appear unrelated, and the runtime tests pass locally for me.

@gharris1727 gharris1727 merged commit 62998b7 into apache:trunk Mar 6, 2024
1 check failed
@hgeraldino hgeraldino deleted the workersinktasktest-batch3 branch March 10, 2024 13:21
testn pushed a commit to testn/kafka that referenced this pull request Mar 15, 2024
clolov pushed a commit to clolov/kafka that referenced this pull request Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
connect tests Test fixes (including flaky tests)
Projects
None yet
3 participants