Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-6727 fix broken Config hashCode() and equals() #4796

Merged

Conversation

big-andy-coates
Copy link
Contributor

@big-andy-coates big-andy-coates commented Mar 29, 2018

Fix for KAFKA-6727.

Current implementation stores reference to entries wrapped in Collections.unmodifiableCollection, which breaks hashCode and equals.

From Java docs

The returned collection does not pass the hashCode and equals operations through to the backing collection, but relies on Object's equals and hashCode methods.

Contribution is my own original work and I license the work to the project under the project's open source license.

Current implementation stores reference to `entries` wrapped in `Collections.unmodifiableCollection`, which breaks `hashCode` and `equals`.
@big-andy-coates
Copy link
Contributor Author

@ijuma @junrao, could I grab a review please?

@omkreddy
Copy link
Contributor

LGTM

@ijuma
Copy link
Contributor

ijuma commented Mar 29, 2018

Good catch. This tells me that we have to stop using UnmodifiableCollection altogether as it's a ticking time bomb. UnmodifiableList and UnmodifiableSet seem to be OK.

@@ -36,14 +37,14 @@
* Create a configuration instance with the provided entries.
*/
public Config(Collection<ConfigEntry> entries) {
this.entries = Collections.unmodifiableCollection(entries);
this.entries = Objects.requireNonNull(entries);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should copy to a list so that we are guaranteed that the collection won't change behind our backs and so that comparison between two Config objects will be valid even if they are constructed using different collection types.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, considering the usage of get below, maybe we should create a Map instead so that we can more easily access an entry by name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about a defensive copy - that would be my standard pattern here. But wanted to minimise my changes.

Give me a mo... I'll switch...

@big-andy-coates
Copy link
Contributor Author

big-andy-coates commented Mar 29, 2018

Yer, UnmodifiableCollection is a bit of a ticking time bomb. But it's OK if used correctly, e.g. as it is now, and saves having to copy collections in accessors. So... it has its place.

BTW, I'd normally test hashCode & equals using Google's EqualsTester. Is there something already like this included in AK? If not, would adding this as a test dependency be an option?

@ijuma @hachikuji @omkreddy code updated inline with review comments.

@ijuma
Copy link
Contributor

ijuma commented Mar 29, 2018

@big-andy-coates I'd argue that what we have now is still not good. :) After all, people can still call equals on the result of entries.

@guozhangwang
Copy link
Contributor

LGTM!

@hachikuji
Copy link

retest this please

@hachikuji
Copy link

@big-andy-coates Note the checkstyle failure. The patch LGTM otherwise.

@big-andy-coates
Copy link
Contributor Author

@ijuma

@big-andy-coates I'd argue that what we have now is still not good. :) After all, people can still call equals on the result of entries.

The docs for this class do not make any guarantees about the equality properties of the return value from entries(), so I don't see this code being 'wrong'. If comparing the return value of entries() with another collection was a common thing, then I'd agree the code code be enhanced to handle this better. Without Guava this would require a defensive copy to be made on each call, or in the constructor. IMHO as I doubt such comparison is a common thing, I don't think it's worth the code complexity / cost.

@big-andy-coates
Copy link
Contributor Author

big-andy-coates commented Apr 9, 2018

retest this please (as the builds have been deleted)

@ijuma
Copy link
Contributor

ijuma commented Apr 9, 2018

@big-andy-coates The class doesn't document equals and hashCode for itself either though. I think it's surprising that the following returns false:

List<ConfigEntry> entries = ...
return new Config(entries).entries().equals(entries);

Don't you?

@big-andy-coates
Copy link
Contributor Author

@ijuma true, but is anyone likely to do this?

Would you prefer if it took a defensive copy?

@big-andy-coates
Copy link
Contributor Author

big-andy-coates commented Apr 9, 2018

@ijuma @hachikuji @omkreddy @guozhangwang changes made - hopefully last review. If you're happy, then can someone please merge.

Thanks all for your time reviewing :D

Copy link

@hachikuji hachikuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@big-andy-coates Thanks. I synced with Ismael offline and had one minor suggestion. If it seems reasonable, we can go ahead and merge.

}

/**
* Configuration entries for a resource.
*/
public Collection<ConfigEntry> entries() {
return entries;
return new ArrayList<>(entries.values());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline with @ijuma. This actually isn't quite enough to fix the example he brought up earlier in all cases. For example, it still doesn't work if you construct Config with a Set for the entries. Given that we can't guarantee equality in this way generally, I'd probably suggest we just revert to using Collections.unmodifiableCollection like you had before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, makes sense. Done.

@big-andy-coates
Copy link
Contributor Author

@hachikuji this should be good to merge now.

@hachikuji
Copy link

LGTM. Will merge as soon as the builds complete.

@hachikuji hachikuji merged commit 432c82d into apache:trunk Apr 16, 2018
@big-andy-coates big-andy-coates deleted the KAFKA-6727_fix_config_hashcode_equals branch April 17, 2018 10:06
jeqo pushed a commit to jeqo/kafka that referenced this pull request May 2, 2018
Reviewers: Manikumar Reddy O <manikumar.reddy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
ying-zheng pushed a commit to ying-zheng/kafka that referenced this pull request Jul 6, 2018
Reviewers: Manikumar Reddy O <manikumar.reddy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants