Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove CompletedFetch type from Fetcher #7228

Merged
merged 3 commits into from Sep 6, 2019

Conversation

@seglo
Copy link
Contributor

commented Aug 21, 2019

Jira issue KAFKA-8822.

The CompletedFetch type is no longer required due to the work merged in PR #6988 (KAFKA-7548). This PR consolidates CompletedFetch into PartitionRecords as discussed in #6988 (comment) with @hachikuji.

@seglo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 24, 2019

Retest this please.

@seglo seglo force-pushed the seglo:seglo/KAFKA-8822 branch 2 times, most recently Aug 24, 2019

@hachikuji hachikuji self-assigned this Aug 27, 2019

@hachikuji
Copy link
Contributor

left a comment

Thanks, nice cleanup! Left a few small comments.

clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java Outdated
@@ -1422,8 +1418,9 @@ private void drain() {
}
}

// TODO: this has no usages. remove?

This comment has been minimized.

Copy link
@hachikuji

hachikuji Aug 28, 2019

Contributor

Yeah, let's remove if it's unused. We can add back if we need it.

clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java Outdated
@@ -1379,10 +1368,12 @@ public static Sensor throttleTimeSensor(Metrics metrics, FetcherMetricsRegistry

private class PartitionRecords {

This comment has been minimized.

Copy link
@hachikuji

hachikuji Aug 28, 2019

Contributor

What do you think about taking the name CompletedFetch here. It seemed a little more descriptive.

This comment has been minimized.

Copy link
@seglo

seglo Aug 28, 2019

Author Contributor

I agree, since it's private to the Fetcher anyway. I've renamed it and a few obvious related variables and javadocs. There are some other things that could be renamed too, but I wanted to see what you thought first.

  • CompletedFetch.isFetched - This sounds a little strange now. isFetched == true when the records are drained which normally happens after it's been fully consumed. We could rename it to isDrained or isConsumed.
  • Fetcher.nextInLineRecords - I assume this was derived from the PartitionRecords type as well, but I'm not sure. It could be renamed to nextInLineCompletedFetch or nextInLineFetch.

This comment has been minimized.

Copy link
@hachikuji

hachikuji Sep 4, 2019

Contributor

Sorry, missed this comment. Both of your suggestions sound good to me. I like isConsumed and nextInLineFetch.

clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java Outdated
FetchResponse.PartitionData<Records> partitionData,
FetchResponseMetricAggregator metricAggregator,
Iterator<? extends RecordBatch> batches,
Long fetchedOffset,

This comment has been minimized.

Copy link
@hachikuji

hachikuji Aug 28, 2019

Contributor

nit: fetchOffset instead of fetchedOffset? I think this is meant to describe the offset from the fetch request.

@seglo seglo force-pushed the seglo:seglo/KAFKA-8822 branch 2 times, most recently Aug 28, 2019

@hachikuji
Copy link
Contributor

left a comment

LGTM. Thanks for the patch! I'll rerun tests one more time before merging since it's been a couple weeks.

@hachikuji

This comment has been minimized.

Copy link
Contributor

commented Sep 4, 2019

retest this please

@seglo

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2019

@hachikuji Thanks. There were a couple more things that could be renamed if you want. See this comment I made earlier in the PR: #7228 (comment)

If you have a moment could you take a quick look at my comments in the benchmark PR? #7221

@hachikuji

This comment has been minimized.

Copy link
Contributor

commented Sep 4, 2019

@seglo Sorry, missed your comment. Your suggestions sound good to me.

@seglo seglo force-pushed the seglo:seglo/KAFKA-8822 branch to b28cc84 Sep 5, 2019

@seglo

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2019

Ok, I think it's ready to go. I'll watch to get a clean test run.

@seglo

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2019

Retest this please.

@hachikuji hachikuji merged commit ad3ccf8 into apache:trunk Sep 6, 2019

3 checks passed

JDK 11 and Scala 2.12 SUCCESS 11967 tests run, 85 skipped, 0 failed.
Details
JDK 11 and Scala 2.13 SUCCESS 11967 tests run, 85 skipped, 0 failed.
Details
JDK 8 and Scala 2.11 SUCCESS 11967 tests run, 85 skipped, 0 failed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.