GEODE-8878: PR clear should also send a lock message to the secondary members. #5950

jinmeiliao · 2021-01-25T20:29:33Z

Thank you for submitting a contribution to Apache Geode.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Has your PR been rebased against the latest commit within the target branch (typically develop)?
Is your initial contribution a single, squashed commit?
Does gradlew build run cleanly?
Have you written or updated unit tests to verify your changes?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:

Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.

… members. * when there are interested clients, we should lock all local buckets first and then distribute lock to other members.

...stributedTest/java/org/apache/geode/cache/query/partitioned/PRClearCreateIndexDUnitTest.java

gesterzhou · 2021-02-01T19:29:02Z

geode-core/src/main/java/org/apache/geode/internal/cache/DistributedClearOperation.java

@@ -211,7 +215,7 @@ protected boolean operateOnRegion(CacheEvent event, ClusterDistributionManager d
      switch (this.clearOp) {
        case OP_CLEAR:
          region.clearRegionLocally((RegionEventImpl) event, false, this.rvv);


why you did not change this line too?

It's a change suggested by IntelliJ, I don't know why it didn't suggest this line. I will change it

Oh, this cast is necessary, the event is of type CacheEvent when passed in.

gesterzhou · 2021-02-03T08:09:27Z

...stributedTest/java/org/apache/geode/cache/query/partitioned/PRClearCreateIndexDUnitTest.java

+      DistributionMessageObserver.setInstance(new MessageObserver());
+      Region<Object, Object> region =
+          ClusterStartupRule.memberStarter.createPartitionRegion("regionA",
+              f -> f.setTotalNumBuckets(1).setRedundantCopies(2));


redundantCopy should be 1 here

is it wrong to use 2? I want to have secondary members to also store data.

1 means 1 secondary. You only have 2 servers

ok, I will make the change

...stributedTest/java/org/apache/geode/cache/query/partitioned/PRClearCreateIndexDUnitTest.java

… lock them

kirklund

As tough as it is, we really need to avoid that Thread.sleep. That's just another new flaky test waiting to run at the wrong time on a slow machine.

jinmeiliao · 2021-02-16T16:47:20Z

As tough as it is, we really need to avoid that Thread.sleep. That's just another new flaky test waiting to run at the wrong time on a slow machine.

@kirklund I've removed the sleep. The test isn't relying on the race to happen. The tests are making sure each member getting the right combination of messages. Please re-review.

kirklund

Approved, but please make the concurrency changes I marked in one inner-class.

kirklund · 2021-02-16T18:46:45Z

...stributedTest/java/org/apache/geode/cache/query/partitioned/PRClearCreateIndexDUnitTest.java

+  }
+
+  private static class MessageObserver extends DistributionMessageObserver {
+    private boolean lock_secondary = false;


The thread invoking the public boolean isXxxxx methods will generally be the JUnit Main thread. The thread invoking beforeProcessMessage will be an internal Geode thread, so you should either make these booleans volatile or change the class to use AtomicBooleans.

It'll probably pass GREEN most of the time as is but this could end up being a source of flakiness.

… members. (#5950)

Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7684: Create messaging class for PR Clear (#4689) * Added new message class and test Co-authored-by: Benjamin Ross <bross@pivotal.io> Co-authored-by: Donal Evans <doevans@pivotal.io> GEODE-7682: add PR.clear API (#4755) * GEODE-7683: introduce BR.cmnClearRegion Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> PR.clear's event id should be created and used in BR (#4805) * GEODE-7857: PR.clear's event id should be created and used in BR GEODE-7912: cacheWriter should be triggered when PR.clear (#4882) Co-authored-by: Anil <agingade@pivotal.io> Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7983: Clear region writer callbacks should not be invoked for bucket regions (#4954) GEODE-7676: Add PR clear with expiration tests (#4970) Added distributed tests to verify the clear operation on Partitioned Regions works as expected when expiration is configured. - Added unit and distributed tests. - Fixed LocalRegion class to clear the entryExpiryTasks Map whenever the cancelAllEntryExpiryTasks method is invoked. GEODE-7667: Add a 'clear' gfsh command for PR and RR clear (#4818) * Added clear command and modified remove functionality to clear PR Authored-by: Benjamin Ross <bross@pivotal.io> GEODE-7676: Conversion of duration to seconds. GEODE-7894: Moving expiry tasks to AbstractRegion. GEODE-7667: Fixing test to include PR clear help text. GEODE-7678 (2nd PR) - Support for cache-listener and client-notification for Partitioned Region Clear operation (#5124) * GEODE-7678: Add support for cache listener and client notification for PR clear The changes are made to PR clear messaging and locking mechanism to preserve cache-listener and client-events ordering during concurrent cache operation while clear in progress. GEODE-7669 Test coverage for Partitioned Region clear with Overflow enabled (#5189) Authored-by: Jianxia Chen <jchen21@apache.org> GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. (#5208) * GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. Co-authored-by: anilkumar gingade <anil@anilg.local> GEODE-8334: PR.clear should sync with putAll or removeAll on rvvLock (#5365) Co-authored-by: Xiaojian Zhou <zhouxh@vmware.com> Co-authored-by: Anil Gingade <agingade@vmware.com> GEODE-8361: Use Set instead of List to track cleared buckets (#5379) - Refactor PartitionRegionClear to use Set instead of List - Some other changes to remove warnings/alerts from PartitionedRegionClear and PartitionedRegionClearMessage Authored-by: Donal Evans <doevans@vmware.com> GEODE-7670: PR Clear with Concurrent Ops DUnitTests (#4848) Added distributed tests to verify that the clear operation on Partitioned Regions works as expected when there are other concurrent operations happening on the cache (put, putAll, get, remove, removeAll, members added and members removed). GEODE-7680: PR.clear must be successful when interacting with rebalance (#5095) - Added DUnit tests to confirm that clear does not interfere with rebalance or vice versa - Test when member departs during clear/rebalance - Test when member joins during clear/rebalance - Fixed typo in PartitionedRegionClearWithExpirationDUnitTest - Fixed typo in PartitionedRegion - Call assignBucketsToPartitions() on leader colocated region during clear instead of target region Authored-by: Donal Evans <doevans@pivotal.io> GEODE-7846: Adding Stats for Partitioned Region Clear (#5391) Added stats to CachePerfStats for PR Clear - Changed clears to 'regionClears' and 'bucketClears' to differentiate between the number of times the region was cleared and the number of times a bucket was cleared in a PartitionedRegion - Added Local and Total duration stats to record how long clear has been running for a specific region, as well as how long it was spent clearing any specific member GEODE-7672: add dunit test to verify OQL index after PR clear. (#5436) * require rvv lock when create index fix rebase compiling error GEODE-7845 blocking PR region clear if one or more server versions are too old (#5577) - if a server is running an old version when a PR clear is invoked by the client, the client will receive a ServerOperationException with a cause of ServerVersionMismatchException. GEODE-7845: Adding a cleaner simpler test. (#5622) - Changed the test for ServerVersionMismatchException to be more readable. GEODE-7845: Now behaving with clients of various versions. (#5645) - added functionality that would allow the tests to be run using various versions of the clients against and and new versions of the server. GEODE-7858: PR.clear notify client should let the queue holder member to notify (#5677) GEODE-7679 Partitioned Region clear is successful while region is being altered (#5516) GEODE-7675: Partitioned Region clear should be successful when clients are present with subscription enabled (#5727) GEODE-8771: invalidate should acquire the lock before initIndex (#5823) GEODE-8878: PR clear should also send a lock message to the secondary members. (#5950) GEODE-9132: Minor cleanup of PartitionedRegionClearTest

Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7684: Create messaging class for PR Clear (apache#4689) * Added new message class and test Co-authored-by: Benjamin Ross <bross@pivotal.io> Co-authored-by: Donal Evans <doevans@pivotal.io> GEODE-7682: add PR.clear API (apache#4755) * GEODE-7683: introduce BR.cmnClearRegion Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> PR.clear's event id should be created and used in BR (apache#4805) * GEODE-7857: PR.clear's event id should be created and used in BR GEODE-7912: cacheWriter should be triggered when PR.clear (apache#4882) Co-authored-by: Anil <agingade@pivotal.io> Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7983: Clear region writer callbacks should not be invoked for bucket regions (apache#4954) GEODE-7676: Add PR clear with expiration tests (apache#4970) Added distributed tests to verify the clear operation on Partitioned Regions works as expected when expiration is configured. - Added unit and distributed tests. - Fixed LocalRegion class to clear the entryExpiryTasks Map whenever the cancelAllEntryExpiryTasks method is invoked. GEODE-7667: Add a 'clear' gfsh command for PR and RR clear (apache#4818) * Added clear command and modified remove functionality to clear PR Authored-by: Benjamin Ross <bross@pivotal.io> GEODE-7676: Conversion of duration to seconds. GEODE-7894: Moving expiry tasks to AbstractRegion. GEODE-7667: Fixing test to include PR clear help text. GEODE-7678 (2nd PR) - Support for cache-listener and client-notification for Partitioned Region Clear operation (apache#5124) * GEODE-7678: Add support for cache listener and client notification for PR clear The changes are made to PR clear messaging and locking mechanism to preserve cache-listener and client-events ordering during concurrent cache operation while clear in progress. GEODE-7669 Test coverage for Partitioned Region clear with Overflow enabled (apache#5189) Authored-by: Jianxia Chen <jchen21@apache.org> GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. (apache#5208) * GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. Co-authored-by: anilkumar gingade <anil@anilg.local> GEODE-8334: PR.clear should sync with putAll or removeAll on rvvLock (apache#5365) Co-authored-by: Xiaojian Zhou <zhouxh@vmware.com> Co-authored-by: Anil Gingade <agingade@vmware.com> GEODE-8361: Use Set instead of List to track cleared buckets (apache#5379) - Refactor PartitionRegionClear to use Set instead of List - Some other changes to remove warnings/alerts from PartitionedRegionClear and PartitionedRegionClearMessage Authored-by: Donal Evans <doevans@vmware.com> GEODE-7670: PR Clear with Concurrent Ops DUnitTests (apache#4848) Added distributed tests to verify that the clear operation on Partitioned Regions works as expected when there are other concurrent operations happening on the cache (put, putAll, get, remove, removeAll, members added and members removed). GEODE-7680: PR.clear must be successful when interacting with rebalance (apache#5095) - Added DUnit tests to confirm that clear does not interfere with rebalance or vice versa - Test when member departs during clear/rebalance - Test when member joins during clear/rebalance - Fixed typo in PartitionedRegionClearWithExpirationDUnitTest - Fixed typo in PartitionedRegion - Call assignBucketsToPartitions() on leader colocated region during clear instead of target region Authored-by: Donal Evans <doevans@pivotal.io> GEODE-7846: Adding Stats for Partitioned Region Clear (apache#5391) Added stats to CachePerfStats for PR Clear - Changed clears to 'regionClears' and 'bucketClears' to differentiate between the number of times the region was cleared and the number of times a bucket was cleared in a PartitionedRegion - Added Local and Total duration stats to record how long clear has been running for a specific region, as well as how long it was spent clearing any specific member GEODE-7672: add dunit test to verify OQL index after PR clear. (apache#5436) * require rvv lock when create index fix rebase compiling error GEODE-7845 blocking PR region clear if one or more server versions are too old (apache#5577) - if a server is running an old version when a PR clear is invoked by the client, the client will receive a ServerOperationException with a cause of ServerVersionMismatchException. GEODE-7845: Adding a cleaner simpler test. (apache#5622) - Changed the test for ServerVersionMismatchException to be more readable. GEODE-7845: Now behaving with clients of various versions. (apache#5645) - added functionality that would allow the tests to be run using various versions of the clients against and and new versions of the server. GEODE-7858: PR.clear notify client should let the queue holder member to notify (apache#5677) GEODE-7679 Partitioned Region clear is successful while region is being altered (apache#5516) GEODE-7675: Partitioned Region clear should be successful when clients are present with subscription enabled (apache#5727) GEODE-8771: invalidate should acquire the lock before initIndex (apache#5823) GEODE-8878: PR clear should also send a lock message to the secondary members. (apache#5950) GEODE-9132: Minor cleanup of PartitionedRegionClearTest

Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7684: Create messaging class for PR Clear (#4689) * Added new message class and test Co-authored-by: Benjamin Ross <bross@pivotal.io> Co-authored-by: Donal Evans <doevans@pivotal.io> GEODE-7682: add PR.clear API (#4755) * GEODE-7683: introduce BR.cmnClearRegion Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> PR.clear's event id should be created and used in BR (#4805) * GEODE-7857: PR.clear's event id should be created and used in BR GEODE-7912: cacheWriter should be triggered when PR.clear (#4882) Co-authored-by: Anil <agingade@pivotal.io> Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7983: Clear region writer callbacks should not be invoked for bucket regions (#4954) GEODE-7676: Add PR clear with expiration tests (#4970) Added distributed tests to verify the clear operation on Partitioned Regions works as expected when expiration is configured. - Added unit and distributed tests. - Fixed LocalRegion class to clear the entryExpiryTasks Map whenever the cancelAllEntryExpiryTasks method is invoked. GEODE-7667: Add a 'clear' gfsh command for PR and RR clear (#4818) * Added clear command and modified remove functionality to clear PR Authored-by: Benjamin Ross <bross@pivotal.io> GEODE-7676: Conversion of duration to seconds. GEODE-7894: Moving expiry tasks to AbstractRegion. GEODE-7667: Fixing test to include PR clear help text. GEODE-7678 (2nd PR) - Support for cache-listener and client-notification for Partitioned Region Clear operation (#5124) * GEODE-7678: Add support for cache listener and client notification for PR clear The changes are made to PR clear messaging and locking mechanism to preserve cache-listener and client-events ordering during concurrent cache operation while clear in progress. GEODE-7669 Test coverage for Partitioned Region clear with Overflow enabled (#5189) Authored-by: Jianxia Chen <jchen21@apache.org> GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. (#5208) * GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. Co-authored-by: anilkumar gingade <anil@anilg.local> GEODE-8334: PR.clear should sync with putAll or removeAll on rvvLock (#5365) Co-authored-by: Xiaojian Zhou <zhouxh@vmware.com> Co-authored-by: Anil Gingade <agingade@vmware.com> GEODE-8361: Use Set instead of List to track cleared buckets (#5379) - Refactor PartitionRegionClear to use Set instead of List - Some other changes to remove warnings/alerts from PartitionedRegionClear and PartitionedRegionClearMessage Authored-by: Donal Evans <doevans@vmware.com> GEODE-7670: PR Clear with Concurrent Ops DUnitTests (#4848) Added distributed tests to verify that the clear operation on Partitioned Regions works as expected when there are other concurrent operations happening on the cache (put, putAll, get, remove, removeAll, members added and members removed). GEODE-7680: PR.clear must be successful when interacting with rebalance (#5095) - Added DUnit tests to confirm that clear does not interfere with rebalance or vice versa - Test when member departs during clear/rebalance - Test when member joins during clear/rebalance - Fixed typo in PartitionedRegionClearWithExpirationDUnitTest - Fixed typo in PartitionedRegion - Call assignBucketsToPartitions() on leader colocated region during clear instead of target region Authored-by: Donal Evans <doevans@pivotal.io> GEODE-7846: Adding Stats for Partitioned Region Clear (#5391) Added stats to CachePerfStats for PR Clear - Changed clears to 'regionClears' and 'bucketClears' to differentiate between the number of times the region was cleared and the number of times a bucket was cleared in a PartitionedRegion - Added Local and Total duration stats to record how long clear has been running for a specific region, as well as how long it was spent clearing any specific member GEODE-7672: add dunit test to verify OQL index after PR clear. (#5436) * require rvv lock when create index fix rebase compiling error GEODE-7845 blocking PR region clear if one or more server versions are too old (#5577) - if a server is running an old version when a PR clear is invoked by the client, the client will receive a ServerOperationException with a cause of ServerVersionMismatchException. GEODE-7845: Adding a cleaner simpler test. (#5622) - Changed the test for ServerVersionMismatchException to be more readable. GEODE-7845: Now behaving with clients of various versions. (#5645) - added functionality that would allow the tests to be run using various versions of the clients against and and new versions of the server. GEODE-7858: PR.clear notify client should let the queue holder member to notify (#5677) GEODE-7679 Partitioned Region clear is successful while region is being altered (#5516) GEODE-7675: Partitioned Region clear should be successful when clients are present with subscription enabled (#5727) GEODE-8771: invalidate should acquire the lock before initIndex (#5823) GEODE-8878: PR clear should also send a lock message to the secondary members. (#5950) GEODE-9132: Minor cleanup of PartitionedRegionClearTest

Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7684: Create messaging class for PR Clear (apache#4689) * Added new message class and test Co-authored-by: Benjamin Ross <bross@pivotal.io> Co-authored-by: Donal Evans <doevans@pivotal.io> GEODE-7682: add PR.clear API (apache#4755) * GEODE-7683: introduce BR.cmnClearRegion Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> PR.clear's event id should be created and used in BR (apache#4805) * GEODE-7857: PR.clear's event id should be created and used in BR GEODE-7912: cacheWriter should be triggered when PR.clear (apache#4882) Co-authored-by: Anil <agingade@pivotal.io> Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7983: Clear region writer callbacks should not be invoked for bucket regions (apache#4954) GEODE-7676: Add PR clear with expiration tests (apache#4970) Added distributed tests to verify the clear operation on Partitioned Regions works as expected when expiration is configured. - Added unit and distributed tests. - Fixed LocalRegion class to clear the entryExpiryTasks Map whenever the cancelAllEntryExpiryTasks method is invoked. GEODE-7667: Add a 'clear' gfsh command for PR and RR clear (apache#4818) * Added clear command and modified remove functionality to clear PR Authored-by: Benjamin Ross <bross@pivotal.io> GEODE-7676: Conversion of duration to seconds. GEODE-7894: Moving expiry tasks to AbstractRegion. GEODE-7667: Fixing test to include PR clear help text. GEODE-7678 (2nd PR) - Support for cache-listener and client-notification for Partitioned Region Clear operation (apache#5124) * GEODE-7678: Add support for cache listener and client notification for PR clear The changes are made to PR clear messaging and locking mechanism to preserve cache-listener and client-events ordering during concurrent cache operation while clear in progress. GEODE-7669 Test coverage for Partitioned Region clear with Overflow enabled (apache#5189) Authored-by: Jianxia Chen <jchen21@apache.org> GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. (apache#5208) * GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. Co-authored-by: anilkumar gingade <anil@anilg.local> GEODE-8334: PR.clear should sync with putAll or removeAll on rvvLock (apache#5365) Co-authored-by: Xiaojian Zhou <zhouxh@vmware.com> Co-authored-by: Anil Gingade <agingade@vmware.com> GEODE-8361: Use Set instead of List to track cleared buckets (apache#5379) - Refactor PartitionRegionClear to use Set instead of List - Some other changes to remove warnings/alerts from PartitionedRegionClear and PartitionedRegionClearMessage Authored-by: Donal Evans <doevans@vmware.com> GEODE-7670: PR Clear with Concurrent Ops DUnitTests (apache#4848) Added distributed tests to verify that the clear operation on Partitioned Regions works as expected when there are other concurrent operations happening on the cache (put, putAll, get, remove, removeAll, members added and members removed). GEODE-7680: PR.clear must be successful when interacting with rebalance (apache#5095) - Added DUnit tests to confirm that clear does not interfere with rebalance or vice versa - Test when member departs during clear/rebalance - Test when member joins during clear/rebalance - Fixed typo in PartitionedRegionClearWithExpirationDUnitTest - Fixed typo in PartitionedRegion - Call assignBucketsToPartitions() on leader colocated region during clear instead of target region Authored-by: Donal Evans <doevans@pivotal.io> GEODE-7846: Adding Stats for Partitioned Region Clear (apache#5391) Added stats to CachePerfStats for PR Clear - Changed clears to 'regionClears' and 'bucketClears' to differentiate between the number of times the region was cleared and the number of times a bucket was cleared in a PartitionedRegion - Added Local and Total duration stats to record how long clear has been running for a specific region, as well as how long it was spent clearing any specific member GEODE-7672: add dunit test to verify OQL index after PR clear. (apache#5436) * require rvv lock when create index fix rebase compiling error GEODE-7845 blocking PR region clear if one or more server versions are too old (apache#5577) - if a server is running an old version when a PR clear is invoked by the client, the client will receive a ServerOperationException with a cause of ServerVersionMismatchException. GEODE-7845: Adding a cleaner simpler test. (apache#5622) - Changed the test for ServerVersionMismatchException to be more readable. GEODE-7845: Now behaving with clients of various versions. (apache#5645) - added functionality that would allow the tests to be run using various versions of the clients against and and new versions of the server. GEODE-7858: PR.clear notify client should let the queue holder member to notify (apache#5677) GEODE-7679 Partitioned Region clear is successful while region is being altered (apache#5516) GEODE-7675: Partitioned Region clear should be successful when clients are present with subscription enabled (apache#5727) GEODE-8771: invalidate should acquire the lock before initIndex (apache#5823) GEODE-8878: PR clear should also send a lock message to the secondary members. (apache#5950) GEODE-9132: Minor cleanup of PartitionedRegionClearTest

Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7684: Create messaging class for PR Clear (#4689) * Added new message class and test Co-authored-by: Benjamin Ross <bross@pivotal.io> Co-authored-by: Donal Evans <doevans@pivotal.io> GEODE-7682: add PR.clear API (#4755) * GEODE-7683: introduce BR.cmnClearRegion Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> PR.clear's event id should be created and used in BR (#4805) * GEODE-7857: PR.clear's event id should be created and used in BR GEODE-7912: cacheWriter should be triggered when PR.clear (#4882) Co-authored-by: Anil <agingade@pivotal.io> Co-authored-by: Xiaojian Zhou <gzhou@pivotal.io> GEODE-7983: Clear region writer callbacks should not be invoked for bucket regions (#4954) GEODE-7676: Add PR clear with expiration tests (#4970) Added distributed tests to verify the clear operation on Partitioned Regions works as expected when expiration is configured. - Added unit and distributed tests. - Fixed LocalRegion class to clear the entryExpiryTasks Map whenever the cancelAllEntryExpiryTasks method is invoked. GEODE-7667: Add a 'clear' gfsh command for PR and RR clear (#4818) * Added clear command and modified remove functionality to clear PR Authored-by: Benjamin Ross <bross@pivotal.io> GEODE-7676: Conversion of duration to seconds. GEODE-7894: Moving expiry tasks to AbstractRegion. GEODE-7667: Fixing test to include PR clear help text. GEODE-7678 (2nd PR) - Support for cache-listener and client-notification for Partitioned Region Clear operation (#5124) * GEODE-7678: Add support for cache listener and client notification for PR clear The changes are made to PR clear messaging and locking mechanism to preserve cache-listener and client-events ordering during concurrent cache operation while clear in progress. GEODE-7669 Test coverage for Partitioned Region clear with Overflow enabled (#5189) Authored-by: Jianxia Chen <jchen21@apache.org> GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. (#5208) * GEODE-8173: Add unit test (coverage) for PartitionedRegionClear class. Co-authored-by: anilkumar gingade <anil@anilg.local> GEODE-8334: PR.clear should sync with putAll or removeAll on rvvLock (#5365) Co-authored-by: Xiaojian Zhou <zhouxh@vmware.com> Co-authored-by: Anil Gingade <agingade@vmware.com> GEODE-8361: Use Set instead of List to track cleared buckets (#5379) - Refactor PartitionRegionClear to use Set instead of List - Some other changes to remove warnings/alerts from PartitionedRegionClear and PartitionedRegionClearMessage Authored-by: Donal Evans <doevans@vmware.com> GEODE-7670: PR Clear with Concurrent Ops DUnitTests (#4848) Added distributed tests to verify that the clear operation on Partitioned Regions works as expected when there are other concurrent operations happening on the cache (put, putAll, get, remove, removeAll, members added and members removed). GEODE-7680: PR.clear must be successful when interacting with rebalance (#5095) - Added DUnit tests to confirm that clear does not interfere with rebalance or vice versa - Test when member departs during clear/rebalance - Test when member joins during clear/rebalance - Fixed typo in PartitionedRegionClearWithExpirationDUnitTest - Fixed typo in PartitionedRegion - Call assignBucketsToPartitions() on leader colocated region during clear instead of target region Authored-by: Donal Evans <doevans@pivotal.io> GEODE-7846: Adding Stats for Partitioned Region Clear (#5391) Added stats to CachePerfStats for PR Clear - Changed clears to 'regionClears' and 'bucketClears' to differentiate between the number of times the region was cleared and the number of times a bucket was cleared in a PartitionedRegion - Added Local and Total duration stats to record how long clear has been running for a specific region, as well as how long it was spent clearing any specific member GEODE-7672: add dunit test to verify OQL index after PR clear. (#5436) * require rvv lock when create index fix rebase compiling error GEODE-7845 blocking PR region clear if one or more server versions are too old (#5577) - if a server is running an old version when a PR clear is invoked by the client, the client will receive a ServerOperationException with a cause of ServerVersionMismatchException. GEODE-7845: Adding a cleaner simpler test. (#5622) - Changed the test for ServerVersionMismatchException to be more readable. GEODE-7845: Now behaving with clients of various versions. (#5645) - added functionality that would allow the tests to be run using various versions of the clients against and and new versions of the server. GEODE-7858: PR.clear notify client should let the queue holder member to notify (#5677) GEODE-7679 Partitioned Region clear is successful while region is being altered (#5516) GEODE-7675: Partitioned Region clear should be successful when clients are present with subscription enabled (#5727) GEODE-8771: invalidate should acquire the lock before initIndex (#5823) GEODE-8878: PR clear should also send a lock message to the secondary members. (#5950) GEODE-9132: Minor cleanup of PartitionedRegionClearTest

jinmeiliao force-pushed the invalidIndex branch 2 times, most recently from 036f0a5 to 64d8785 Compare January 27, 2021 22:02

jinmeiliao changed the title ~~Invalid index~~ GEODE-8878: PR clear should also send a lock message to the secondary members. Jan 27, 2021

jinmeiliao requested review from agingade, gesterzhou, kirklund, jchen21, mhansonp, demery-pivotal and dschneider-pivotal January 27, 2021 22:03

jinmeiliao marked this pull request as ready for review January 27, 2021 22:04

GEODE-8878: PR clear should also send a lock message to the secondary…

f88246d

… members. * when there are interested clients, we should lock all local buckets first and then distribute lock to other members.

jinmeiliao force-pushed the invalidIndex branch from 64d8785 to f88246d Compare January 29, 2021 19:09

kirklund reviewed Jan 29, 2021

View reviewed changes

...stributedTest/java/org/apache/geode/cache/query/partitioned/PRClearCreateIndexDUnitTest.java Show resolved Hide resolved

gesterzhou reviewed Feb 1, 2021

View reviewed changes

gesterzhou reviewed Feb 3, 2021

View reviewed changes

...stributedTest/java/org/apache/geode/cache/query/partitioned/PRClearCreateIndexDUnitTest.java Show resolved Hide resolved

review update

e4328f2

dschneider-pivotal approved these changes Feb 4, 2021

View reviewed changes

always have the primary buckets send messages to secondary buckets to…

267930b

… lock them

jinmeiliao requested review from dschneider-pivotal, gesterzhou and kirklund February 10, 2021 21:05

kirklund requested changes Feb 11, 2021

View reviewed changes

gesterzhou approved these changes Feb 11, 2021

View reviewed changes

remove sleep and update the comments

c243de5

jinmeiliao requested a review from kirklund February 11, 2021 19:06

kirklund approved these changes Feb 16, 2021

View reviewed changes

review changes

3160370

jinmeiliao force-pushed the invalidIndex branch from ff89cce to 3160370 Compare February 16, 2021 20:55

jinmeiliao merged commit 5156246 into apache:feature/GEODE-7665 Feb 16, 2021

jinmeiliao deleted the invalidIndex branch February 16, 2021 21:10

nabarunnag pushed a commit that referenced this pull request Apr 5, 2021

GEODE-8878: PR clear should also send a lock message to the secondary…

4831647

… members. (#5950)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GEODE-8878: PR clear should also send a lock message to the secondary members. #5950

GEODE-8878: PR clear should also send a lock message to the secondary members. #5950

jinmeiliao commented Jan 25, 2021

gesterzhou Feb 1, 2021

jinmeiliao Feb 1, 2021

jinmeiliao Feb 4, 2021

gesterzhou Feb 3, 2021

jinmeiliao Feb 3, 2021

gesterzhou Feb 3, 2021

jinmeiliao Feb 3, 2021

kirklund left a comment

jinmeiliao commented Feb 16, 2021

kirklund left a comment

kirklund Feb 16, 2021

GEODE-8878: PR clear should also send a lock message to the secondary members. #5950

GEODE-8878: PR clear should also send a lock message to the secondary members. #5950

Conversation

jinmeiliao commented Jan 25, 2021

For all changes:

Note:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kirklund left a comment

Choose a reason for hiding this comment

jinmeiliao commented Feb 16, 2021

kirklund left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment