New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent reads of events that have not been replicated #1448

Merged
merged 1 commit into from Dec 14, 2017

Conversation

2 participants
@hayley-jean
Contributor

hayley-jean commented Sep 29, 2017

Only update the index once a commit has been replicated in a cluster

Introduces ReplicationCheckpoint to track the position of the last committed event.

Introduces IndexCommitterService to replace the IndexCommitter in StorageChaser.
This tracks pending transactions from the StorageChaser and CommitAcks from the cluster.
When enough commit acks have been received, the index and the replication checkpoint is updated.
If a CommitAck is received on a non-master node, the event is immediately considered replicated.
If a master node changes to a non-master, all pending commits are considered replicated.

Update TwoPhaseRequestManagers to listen for CommitReplicated messages rather than CommitAck messages.
This means that a write is only complete once the commit has been replicated and the index updated.

Update AllReader to only read up to the ReplicationCheckpoint

Also refactored tests using TFChunkDbConfig to use a constructor from a helper method to make future constructor updates easier.

@hayley-jean

This comment has been minimized.

Show comment
Hide comment
@hayley-jean

hayley-jean Nov 8, 2017

Contributor

@gregoryyoung, @pgermishuys Would you mind having a look at this and seeing if there's any potential issues or warnings with this approach?

Contributor

hayley-jean commented Nov 8, 2017

@gregoryyoung, @pgermishuys Would you mind having a look at this and seeing if there's any potential issues or warnings with this approach?

Only update the index once a commit has been replicated in a cluster
Introduces ReplicationCheckpoint to track the position of the last committed event.

Introduces IndexCommitterService to replace the IndexCommitter in StorageChaser.
This tracks pending transactions from the StorageChaser and CommitAcks from the cluster.
When enough commit acks have been received, the index and the replication checkpoint is updated.
If a CommitAck is received on a non-master node, the event is immediately considered replicated.
If a master node changes to a non-master, all pending commits are considered replicated.

Update TwoPhaseRequestManagers to listen for CommitReplicated messages rather than CommitAck messages.
This means that a write is only complete once the commit has been replicated and the index updated.

Update AllReader to only read up to the ReplicationCheckpoint

Update parked event and list projections tests.

@shaan1337 shaan1337 merged commit 5d58ede into release-v4.0.4 Dec 14, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
wercker/build-mono4 Wercker pipeline passed
Details

@shaan1337 shaan1337 deleted the dont-read-before-replication branch Dec 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment