Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unnecessary data copy on index add #13965

Closed

Conversation

TomaszGaweda
Copy link
Contributor

For the the situation when the index already exists, then there is no need to fill it again with the same data

Closes #13964

@TomaszGaweda
Copy link
Contributor Author

@mmedenjak @taburet Can you please add me to CloudBees or paste here what the error was?

@mmedenjak
Copy link
Contributor

mmedenjak commented Oct 17, 2018

Hi @TomaszGaweda , these are the failed tests:

com.hazelcast.client.map.ClientPagingPredicateTest.testIssue3047
com.hazelcast.client.map.ClientPagingPredicateTest.mapPagingPredicateEmployeeObjectWithOrderedIndexSmallTest
com.hazelcast.client.map.ClientPagingPredicateTest.mapPagingPredicateEmployeeObjectWithOrderedIndexLargeTest
com.hazelcast.map.PagingPredicateTest.testPagingWithFilteringAndComparatorAndIndex

I've checked the first one and it seems to fail consistently.

@mmedenjak
Copy link
Contributor

Can you also please follow the instructions to sign and send the Hazelcast Contributor Agreement?

@TomaszGaweda
Copy link
Contributor Author

@mmedenjak Sure, I will sign it, my boss just wants to check this by lawyer to be sure :) One test is broken, I probably have fix and will commit shortly

@taburet
Copy link
Contributor

taburet commented Oct 22, 2018

@TomaszGaweda the tests are failing because of the so called global indexes, which share their index definitions across partitions, so if a global index definition is present that doesn't mean a corresponding index is populated by all partitions. Apparently, to fix that we need to track a set of indexed partitions.

Also, I have discovered another flaw in the distributed map proxies implementation: looks like we are rebuilding all indexes (configured using an xml config) when a new member joins a cluster even if there are no manual calls to addIndex. See com.hazelcast.map.impl.proxy.MapProxySupport#initialize and com.hazelcast.map.impl.proxy.MapProxySupport#initializeIndexes.

If you don't mind, I will take over the issue and fix both problems.

@TomaszGaweda
Copy link
Contributor Author

@taburet Sorry for the late response. It would be great if you can take over this issue, I'm not feeling I can easilly fix the second issue you've written. For the issue with tests, I've tried to add counter to Indexes class that checks if all partitions are migrated, seems to work, but I didn't have time to check it more than rerun test

@taburet
Copy link
Contributor

taburet commented Dec 10, 2018

Superseded by #13984, closing.

@taburet taburet closed this Dec 10, 2018
@mmedenjak mmedenjak added the Source: Community PR or issue was opened by a community user label Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unnecessary data copy on index add
3 participants