New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PUBDEV-6319: Fix a race condition in group-by implementation #3725
Conversation
1cfa766
to
41b0ca1
Compare
41b0ca1
to
96f51db
Compare
Fixes a race condition demonstrated in #3726
This makes sure data is not duplicated when serialized, this can still have memory impact because keys are not guaranteed to stay the same as the values.
96f51db
to
af44c23
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So you just use a Hashset instead but with it delegating all works to the original nonblockingHashMap iced. Great! I would like to do a timing test after this is merge.
Do you want to merge this into rel-yau? Need to fix it there too, no? Wendy |
Exactly... that prevents users to use the HashMap the wrong way. But that is just a syntactic sugar; the core of the fix is in the first comment where I swapped working with keys (that can change) to values that are immutable once inserted. |
Yes, absolutely correct - I will merge into master first and when appropriate backport to a desired Yau fix release. I didn't see any changes in performance in my tests but I will reach out to you and we can coordinate tests with your test suite. |
@wendycwong I couldn't do full tests yet but multinode tests show the performance is definitely not worse (times in seconds): New: Old: This could be attributed to a faster serialization - we are only serializing the values before we were serializing both keys and values. |
@michalk: Nice side effect. You fixed the main problem and then gain more speed along the way. Wish I could do that! |
No description provided.