Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CRDT: "Direct Replication" / at-Consistency writes #27

Closed
ktoso opened this issue Aug 22, 2019 · 0 comments · Fixed by #117
Closed

CRDT: "Direct Replication" / at-Consistency writes #27

ktoso opened this issue Aug 22, 2019 · 0 comments · Fixed by #117
Assignees
@yim-lee
Labels
2 - pick next If looking for work, pick this ticket :-)
Milestone
0.1.0

Comments

@ktoso
Copy link
Member

ktoso commented Aug 22, 2019

This is about implementing the direct replication for CRDTs when they do write(consistency: .all) or similar.

We should:

  • send the write to N random members
  • complete the write once we get N acks back and fail it otherwise

This is enough to have CRDTs "work" even without gossip, if we always were to write to all nodes, and no cluster membership changes.

We will follow up with gossip in separate ticket.
@ktoso ktoso added 1 - triaged Ticket makes sense and is well defined, ready to be worked on 2 - pick next If looking for work, pick this ticket :-) t:crdt labels Aug 22, 2019
@ktoso ktoso added this to the 0.1.0 milestone Aug 22, 2019
@ktoso ktoso mentioned this issue Aug 22, 2019
Closed
@yim-lee yim-lee mentioned this issue Sep 1, 2019
Closed
3 tasks
@ktoso ktoso mentioned this issue Sep 4, 2019
Merged
ktoso pushed a commit that referenced this issue Sep 6, 2019
@yim-lee @ktoso
CRDT Serialization
2fef7d3 
Motivation:
For direct replication (#27) we need to send CRDTs to remote replicators, therefore they need to be serializable.

Modifications:
- Define protobufs for CRDTs and conform to `InternalProtobufRepresentable`.
- Define protobufs for `CRDT.Replicator.RemoteCommand`.
- Special handling in `Serialization` for CRDT serializers.

Result:
Be able to send `CRDT.Replicator.RemoteCommand` message to remote replicators.
This was referenced Sep 15, 2019
Merged
Merged
ktoso pushed a commit that referenced this issue Sep 16, 2019
@yim-lee @ktoso
CRDT replication serialization (#118)
752a5f3 
* CRDT replication serialization

Motivation:
`CRDT.Replicator.RemoteCommand` messages need to be serialized for replication. Groundwork has already been done in previous PRs; this PR wraps it up.

Modifications:
- Serialization for all `CRDT.Replicator.RemoteCommand` messages.
- Per [protobuf style guide](https://developers.google.com/protocol-buffers/docs/style), the zero value enum should be "unspecified".

Results:
`CRDT.Replicator.RemoteCommand` messages ready for replication.

Part of #27

* Unformat .pb.swift files

* Remove prefix for 'unspecified'
@ktoso ktoso added 3 - in progress Ticket is being worked on and removed 1 - triaged Ticket makes sense and is well defined, ready to be worked on labels Sep 16, 2019
yim-lee added a commit that referenced this issue Sep 23, 2019
@yim-lee
CRDT direct replication (#117)
652caa8 
Motivation:
Direct replication of local CRDT changes to remote nodes as `LocalCommand`s get processed.

Changes:
- `CRDT.Replicator.Shell` subscribes to cluster membership events in order to keep track of remote replicators.
- Logic to determine remote confirmations/acks needed for a given `OperationConsistency`.
- Construct and send `RemoteCommand` for `LocalCommand` `.write`, `.read`, and `.delete` as part of direct replication. In case of `read`, this is to read CRDT from remote nodes and update local copy.
- Add tracelog to all switch cases (per [feedback](#118 (comment)))
- Rename `.failed` enum case to `.failure` to be consistent with NIO and `Result`.

Result:
Resolves #27.
@avolokhov avolokhov mentioned this issue Mar 8, 2020
Closed
@ktoso ktoso mentioned this issue Jun 25, 2020
Closed
@ktoso ktoso removed the 3 - in progress Ticket is being worked on label May 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yim-lee yim-lee

Labels
2 - pick next If looking for work, pick this ticket :-)
Projects
None yet
Milestone
0.1.0
Development

Successfully merging a pull request may close this issue.

CRDT direct replication
2 participants
@ktoso @yim-lee