Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KAFKA-6005: Reject JoinGroup request from first member with empty protocol type/protocol list #3957

Closed
wants to merge 4 commits into from
Closed

KAFKA-6005: Reject JoinGroup request from first member with empty protocol type/protocol list #3957

wants to merge 4 commits into from

Conversation

omkreddy
Copy link
Contributor

No description provided.

@omkreddy
Copy link
Contributor Author

omkreddy commented Sep 25, 2017 •
edited

@hachikuji if the first group member joins with empty partition.assignment.strategy, then the group won't allow any other members with valid protocols.
Also currently, we are allowing Consumer clients to set null/empty partition.assignment.strategy. should we allow this?

@omkreddy
Copy link
Contributor Author

@hachikuji pinging for review

hachikuji
hachikuji reviewed Sep 27, 2017
View reviewed changes
Copy link
Contributor

@hachikuji hachikuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch. Left a couple comments.

core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala Outdated
@@ -146,6 +146,9 @@ class GroupCoordinator(val brokerId: Int,
if (!group.is(Empty) && (!group.protocolType.contains(protocolType) || !group.supportsProtocols(protocols.map(_._1).toSet))) {
// if the new member does not support the group protocol, reject it
responseCallback(joinError(memberId, Errors.INCONSISTENT_GROUP_PROTOCOL))
} else if (group.is(Empty) && protocols.isEmpty ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should also check whether protocolType is non-empty?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a non-empty check on protocolType

core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala Outdated
@@ -146,6 +146,9 @@ class GroupCoordinator(val brokerId: Int,
if (!group.is(Empty) && (!group.protocolType.contains(protocolType) || !group.supportsProtocols(protocols.map(_._1).toSet))) {
// if the new member does not support the group protocol, reject it
responseCallback(joinError(memberId, Errors.INCONSISTENT_GROUP_PROTOCOL))
} else if (group.is(Empty) && protocols.isEmpty ) {
//reject if first member with empty group protocol
responseCallback(joinError(memberId, Errors.INCONSISTENT_GROUP_PROTOCOL))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error code seems less than ideal, though I'm not sure there's a better alternative. Maybe we can at least extend the description in Errors to cover this case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the description of INCONSISTENT_GROUP_PROTOCOL error

@hachikuji
Copy link
Contributor

@omkreddy By the way, did someone hit a problem with the current behavior or did you just notice the missing validation?

@omkreddy
Copy link
Contributor Author

@hachikuji apparently one of our customers tried with null partition.assignment.strategy. I am not sure why they tried :)

@hachikuji
Copy link
Contributor

@omkreddy That's interesting. Seems like a validation gap on the client as well. Maybe when the user calls one of the subscribe APIs, we can add an assertion that there is at least one configured partition assigner?

@omkreddy
Copy link
Contributor Author

@hachikuji added a client-side validation check. Updated the PR

omkreddy
omkreddy commented Sep 28, 2017
View reviewed changes
clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java Outdated

if (assignors.isEmpty())
throw new IllegalArgumentException("Must specify at least one partition assigner");

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to use ConfigException here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's a good question. IllegalArgumentException doesn't feel quite right. Another option would be IllegalStateException. Also, we should probably mention the configuration property in the exception message.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hachikuji Updated the PR. Please check.

hachikuji
hachikuji approved these changes Oct 2, 2017
View reviewed changes
Copy link
Contributor

@hachikuji hachikuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor comments. The rest LGTM.

clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java Outdated

if (assignors.isEmpty())
throw new IllegalStateException("Must configure at least one partition assigner class name to" +
" partition.assignment.strategy config");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can we reference the property in ConsumerConfig instead? Also, not sure it's worth it, but we could factor out a function, say throwIfNoAssignorsConfigured to avoid duplicating the comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good..Updated PR

clients/src/main/java/org/apache/kafka/common/protocol/Errors.java Outdated
@@ -266,7 +266,8 @@ public ApiException build(String message) {
}
}),
INCONSISTENT_GROUP_PROTOCOL(23,
"The group member's supported protocols are incompatible with those of existing members.",
"The group member's supported protocols are incompatible with those of existing members" +
" or first group member tried to join with empty protocol.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two cases, right? The protocol type might be empty, or the list of protocols might be empty.

@hachikuji
Copy link
Contributor

@omkreddy Note the checkstyle error.

@omkreddy
Copy link
Contributor Author

omkreddy commented Oct 2, 2017

@hachikuji ahh..sorry..fixed the check style issue.

@hachikuji
Copy link
Contributor

@omkreddy Since this is really a bug fix, would you mind creating a JIRA for this issue before I merge?

@omkreddy omkreddy changed the title MIROR: Reject JoinGroup request from first member with empty partition.assignment.strategy KAFKA-6005: Reject JoinGroup request from first member with empty partition.assignment.strategy Oct 3, 2017
@omkreddy
Copy link
Contributor Author

omkreddy commented Oct 3, 2017

Raised JIRA for this : https://issues.apache.org/jira/browse/KAFKA-6005

@omkreddy omkreddy changed the title KAFKA-6005: Reject JoinGroup request from first member with empty partition.assignment.strategy KAFKA-6005: Reject JoinGroup request from first member with empty protocol type/protocol list Oct 3, 2017
@hachikuji
Copy link
Contributor

Thanks for the patch. Merging to trunk.

@asfgit asfgit closed this in 42b3565 Oct 3, 2017
@omkreddy omkreddy deleted the JOIN-GROUP-EMPTY-PROTOCOL branch July 3, 2018 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hachikuji hachikuji hachikuji approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@omkreddy @hachikuji