Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report WAN sync progress based on acknowledgement #15221

Merged

Conversation

@blazember
Copy link
Contributor

blazember commented Jun 26, 2019

The change makes WAN synchronization to report progress based on target acknowledgments instead of reporting based on enqueueing WAN synchronization events in the local cluster. This way the user sees progress only when the events reached the target cluster. Synchronization events are getting acknowledged when the WAN operation carrying the event gets enqueued in the target cluster. This still leaves it possible that in the target cluster the synchronization event won't get processed. WAN operations trigger merge operations in the target cluster that possibly ends in remote invocations. Since WAN operations are throttled by acknowledgments, acknowledging when the merging is done would make synchronization more time consuming hence the acknowledgment is not changed.

As part of this change, a synchronization UUID has been introduced that uniquely identifies a WAN synchronization (or consistency check) request. This UUID is visible in the logs, diagnostics and is sent to MC as well. The UUID is useful for matching related events when troubleshooting and also helps to distinguish between consecutive or overlapping synchronization processes. As a downside, UUIDs make synchronization events heavier by adding two longs to all of them.

EE PR: hazelcast/hazelcast-enterprise#3058

@blazember blazember added this to the 4.0 milestone Jun 26, 2019
@blazember blazember requested a review from mmedenjak Jun 26, 2019
@blazember blazember self-assigned this Jun 26, 2019
@blazember blazember force-pushed the blazember:4.0/wan-sync-track/wan-sync-track branch from ccf6e65 to 38137a7 Jul 1, 2019
blazember added 2 commits Jun 25, 2019
The change makes WAN synchronization to report progress based on target
acknowledgements instead of reporting based on enqueueing WAN
synchronization events in the local cluster. This way the user sees
progress only when the events reached the target cluster.
Synchronization events are getting acknowledged when the WAN operation
carrying the event gets enqueued in the target cluster. This still leaves
it possible that in the target cluster the synchronization event wont get
processed. WAN operations trigger merge operations in the target cluster
that are possibly end in remote invocations. Since WAN operations are
throttled by acknowedgements, acknowledging when the merging is done would
make synchronization more time consuming hence the acknowledgement is
not changed.

As part of this change, a synchronization UUID has been introduced that
uniquely identifies a WAN synchronization (or consistency check)
request. This UUID is visible in the logs, diagnostics and is sent to MC
as well. The UUID is useful for matching related events when
troubleshooting and also helps distinguishing between consecutive or
overlapping synchronization processes. As a downside, UUIDs make
synchronization events heavier by adding two longs to all of them.
@blazember blazember force-pushed the blazember:4.0/wan-sync-track/wan-sync-track branch from 38137a7 to e9cdb3c Jul 5, 2019
@blazember blazember merged commit 27bf293 into hazelcast:master Jul 25, 2019
1 check passed
1 check passed
default Test PASSed.
Details
@blazember blazember deleted the blazember:4.0/wan-sync-track/wan-sync-track branch Jul 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.