Add support for WAN replication of IMap/ICache evictions (#24941) [5.3.z] #25039
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Backport of: hazelcast#24941 In the current implementation of Hazelcast, evictions are not replicated over WAN to target clusters - this is done intentionally as evictions are purely local operations that usually occur as a last resort to free resources on demand. However, there are circumstances where some users may want to have evictions replicated over WAN to try and maintain more synchronization between 2 clusters (even though this does still not guarantee data consistency using WAN). This commit introduces 2 new `ClusterProperty` entries that allow WAN replication of `IMap` and `ICache` eviction events respectively. This property is disabled by default and must be explicitly enabled as WAN replication of eviction events was purposefully omitted prior to this. Code changes are simple, introducing `publishWanRemove()` calls within `DefaultRecordStore` (for `IMap`) and `AbstractCacheRecordStore` (for `ICache`) - I had originally trialed a solution that introduced these calls within operations (`EvictOperation` etc) to match existing WAN replication mechanics, but due to the mostly-local nature of eviction that is often triggered as a result of other operations (not eviction operations directly), it made more sense to implement at the root of eviction calls, which resides in these record stores. This solution, when enabled, fires WAN replication events for all evictions (expiration, resource constraints, user-invoked, etc) - the reason for this is the use-case for these config options is to attempt to keep data more consistent between 2 clusters over WAN, and so in this scenario (although still not bulletproof by any means), it makes sense to try and replicate all operations that result in mutation (as is the case with all evictions). This commit also includes a regression test that confirms consistency between 2 clusters under ideal circumstances with evictions on the source cluster. Fixes https://hazelcast.atlassian.net/browse/HZ-2619
JamesHazelcast
added
Type: Enhancement
Team: Core
Source: Internal
vbekiaris
approved these changes
Jul 19, 2023
There was a problem hiding this comment.
thanks @JamesHazelcast , just a minor update to the @since version tag needed
hazelcast/src/main/java/com/hazelcast/spi/properties/ClusterProperty.java
Outdated
Show resolved
Hide resolved
hazelcast/src/main/java/com/hazelcast/spi/properties/ClusterProperty.java
Outdated
Show resolved
Hide resolved
Co-authored-by: Vassilis Bekiaris <vbekiaris@gmail.com>
Co-authored-by: Vassilis Bekiaris <vbekiaris@gmail.com>
Good catch, thanks @vbekiaris 👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of: #24941
In the current implementation of Hazelcast, evictions are not replicated over WAN to target clusters - this is done intentionally as evictions are purely local operations that usually occur as a last resort to free resources on demand. However, there are circumstances where some users may want to have evictions replicated over WAN to try and maintain more synchronization between 2 clusters (even though this does still not guarantee data consistency using WAN).
This commit introduces 2 new
ClusterPropertyentries that allow WAN replication ofIMapandICacheeviction events respectively. This property is disabled by default and must be explicitly enabled as WAN replication of eviction events was purposefully omitted prior to this.Code changes are simple, introducing
publishWanRemove()calls withinDefaultRecordStore(forIMap) andAbstractCacheRecordStore(forICache) - I had originally trialed a solution that introduced these calls within operations (EvictOperationetc) to match existing WAN replication mechanics, but due to the mostly-local nature of eviction that is often triggered as a result of other operations (not eviction operations directly), it made more sense to implement at the root of eviction calls, which resides in these record stores.This solution, when enabled, fires WAN replication events for all evictions (expiration, resource constraints, user-invoked, etc) - the reason for this is the use-case for these config options is to attempt to keep data more consistent between 2 clusters over WAN, and so in this scenario (although still not bulletproof by any means), it makes sense to try and replicate all operations that result in mutation (as is the case with all evictions).
This commit also includes a regression test that confirms consistency between 2 clusters under ideal circumstances with evictions on the source cluster.
Fixes https://hazelcast.atlassian.net/browse/HZ-2619
EE PR: https://github.com/hazelcast/hazelcast-enterprise/pull/6260