Server side transforms in Voldemort

Server-side transforms or predicate pushdowns are a way of transforming or filtering the values mapped to a key before transferring it to the client. This functionality is currently unsupported in Voldemort since there is no way to “pass-down” filter queries to server side. In absence of this feature, to filter out values, one needs to fetch it through the ‘get()’ API to client side, and then apply the filter queries there. There is unnecessary data exchanged over the network leading to wastage of network bandwidth and increased latency of the get API.

Similarly for the put() API, if some filters need to be applied on (existing values in the Voldemort store + new values specified through put()), before writing the filtered data back into the store, we need some way to transmit the ‘filter queries’ along with the new values for the key.

If an application needs to support server-side transforms on a store, then it can define a ‘view’ on the store. For filtering in the get() API, the storeToView() method will apply the transforms on the values fetched from the store. For filtering in the put() API, the viewToStore() method will apply the transforms on the old+new data before writing it to the store.

The following changes were made :

View related changes

Both viewToStore() and storeToView() APIs will take in transforms

• public V storeToView(Store<K, S> targetStore, K k, S s, T t) throws UnsupportedViewOperationException;

• public S viewToStore(Store<K, S> targetStore, K k, V v, T t) throws UnsupportedViewOperationException;
  

Store related changes

Change the existing StoreClient<K, V> interface to StoreClient<K, V, T> with following API additions -

• public Versioned get(K key, T transforms);

• public Map<K, Versioned> getAll(Iterable keys, Map<K, T> transforms);

• public void put(K key, V value, T transforms);
  

Make the following API changes to Store<K, V>

• public List<Versioned> get(K key, T transforms) throws VoldemortException;

• public Map<K, List<Versioned>> getAll(Iterable keys, Map<K, T> transforms) throws VoldemortException;

• public void put(K key, Versioned value, T transforms) throws VoldemortException;