Dual Write Functionality

Dual Writes

An application can enable dual writes in order to send shadow traffic to a different dynomite cluster. This is useful in scenarios where a client wishes to eventually migrate to the 'shadow' cluster, for example, to move to a cluster with more capacity, or test the performance of different storage engines under Dynomite.

Setup

Enabling dual writes can be as simple as creating the three properties listed below. This will return an instance of DynoDualWriterClient which inherits from DynoJedisClient, so you will not need to change any application code to use this feature.

Configuration

The following properties are applicable to Dual Writes:

• dyno.{APP}.dualwrite.enabled - true/false
• dyno.{APP}.dualwrite.cluster - name of the shadow cluster
• dyno.{APP}.dualwrite.percentage - amount of traffic to send to the shadow cluster

You can control dual writes solely using these properties or programmatically (see below) or using any combination of the two.

Note that until Archaius 2.x is integrated into the Dyno client property changes require an application restart to take effect.

Controlling dual-write traffic volume

It's often useful to control the amount of shadow traffic for high throughput applications.

To turn dual writes on using properties at 10%

dyno.{APP}.dualwrite.enabled=true
dyno.{APP}.dualwrite.cluster={CLUSTER_NAME}
dyno.{APP}.dualwrite.percentage=10

To turn dual writes on programmatically at 10%.

public DynoDualWriterExample(String clusterName, String shadowClusterName, DiscoveryClient discoveryClient) {

    // Create an instance of the a client with dual-writes enabled
    DynoJedisClient client = new DynoJedisClient.Builder()
                .withApplicationName(clusterName)
                .withDynomiteClusterName(clusterName)
                .withDualWriteClusterName(shadowClusterName)
                .withDiscoveryClient(discoveryClient)
                .build();
                
    // Set the dial to 10%
    client.getDial().setRange(10);
    
    // Perform write operations
    client.set("mykey", "myvalue");
    .
    .
                             
}

The default Dial implementation is based on timestamp, however, applications can have finer grained control of the dial by implementing their own Dial based on the value of their key. The example below shows a custom dial based on key that is a numeric customer id.

public DynoDualWriterExample(String clusterName, String shadowClusterName, DiscoveryClient discoveryClient) {

    // Create a Dial implementation 
    Dial customerDial = new DynoDualWriterClient.Dial() {
        private int range = 25;
        
        @Override
        public boolean isInRange(String key) {
            return (Integer.intValue(key) % 100) < range;            
        }

        @Override
        public boolean isInRange(byte[] key) {
            return false;
        }
        
        @Override
        public void setRange(int range) {
            this.range = range;
        }
    }
    
    // Create an instance of the a client with dual-writes enabled
    DynoJedisClient client = new DynoJedisClient.Builder()
                .withApplicationName(clusterName)
                .withDynomiteClusterName(clusterName)
                .withDualWriteClusterName(dualWriteClusterName)
                .withDial(customerDial)
                .withDiscoveryClient(discoveryClient)
                .build();
                
    // Perform write operations
    client.set("mykey", "myvalue");
    .
    .
                             
}

Switching to a dual-write cluster

To move all traffic from one cluster to another, the following steps can be taken:

1. Enable dual-writes at incrementally higher intervals until the dual-write cluster is taking full traffic, e.g. 10%, 50%, 100%
2. Run in this mode until the TTL has expired for all data in the original cluster.
3. Restart the application with dual-writes disabled, using the dual-write cluster name in the builder value for withDynomiteCluster().

Pipeline operations

Dual-writer client also support pipeline operations. See the pipelining wiki on how to use it.