Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support seamless replication #94

Closed
je-ik opened this issue May 24, 2018 · 1 comment
Closed

Support seamless replication #94

je-ik opened this issue May 24, 2018 · 1 comment

Comments

@je-ik
Copy link
Collaborator

je-ik commented May 24, 2018

Need support for multimaster-multislave replication schemes accross multiple proxima deployments with eventual consistency. Required features:

  • the replication will work on commit logs only - all other consistency and replication will occur inside given proxima deployment
  • easy configuration
  • ability to observe non replicated items in commit log only (defaults to whole commit log in case there is no replication)

Draft of configuration:

 entities {
   original {
     attrA: { scheme: "bytes" }
     attrB: { scheme: "bytes" }
   }
 }
 replication {
   replicate-original-attrA {
     entity: original
     attributes: [ "attrA" ] # can be "*"
     # declare attribute families that will be written when update happens
     targets {
       first {
         // attribute family specifier
       }
       second {
         // attribute family specifier
       }
     }
     # declare single source family that will be read and written to local commit log
     source {
       // attribute family specifier
     }
   }
   replicate-original-all {
     entity: original
     attributes: [ "*" ] # replicate all attributes with some other settings
     targets { ... }
     source { ... }
   }
 }

In order to accomplish this we need to:

  • create implicit attribute for each replicated one (prefixed with _<replication_name>_s_ for source attribute family, _<replication_name>_t_<target_name>_ for target attribute), _<replication_name>_ for direct writes and _<replication_name>_r_ for the replicated attribute containing all replicated data (which will be read instead of original attribute, which becames virtual)
  • relax AttributeProxy so that it can be used only for writes (i.e. attribute is modified only on direct write, read happens on actual attribute) - this will be used to transform writes to attribute X into _<replication_name>_X
  • start replication of _<replication_name>_X into X
  • start replication of _<replication_name>_source_X into X
  • start replication of _<replication_name>_X into each of _<replication_name>_target_<target_name>_X
@je-ik
Copy link
Collaborator Author

je-ik commented Jul 18, 2018

Closed via Closed via datadrivencz#105

@je-ik je-ik closed this as completed Jul 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant