Scatter Gather Operations

[gonzajg]

Hi, I read the wiki about scatter gather operations and how a mset is divided and then weaved to return a single result, but when I try to use it I get a org.apache.commons.lang.NotImplementedException.

Are there any plans to implement this soon? We could really use this feature as mset is essential for us.

Thanks!

[opuneet]

Hi @gonzajg

Yes we don't have this working right now and this feature is coming soon!
Actually, mset in Dynomite will not be scatter gather, it will be something like shard aware micro batches, when you submit the total batch to Dyno.

Reason for the difference is that scatter gather talks to all dynomite nodes, where as something like mset should talk to only nodes that map directly to yours keys.

e.g [a, b, c, d] are your keys, and your mset [a,b,c,d] command may translate to mset [a,d] on one host and mset[b] on another and mset [c] on another, depending on the sharding.

But this is not scatter gather. In any case I can re-use some of the code from scatter gather to actually implement this.

So long story short, coming soon :)

I'll keep the issue open and will resolve when I push the feature.

[gonzajg]

Understood!
I'd always thought this could be done, especially when I saw that mset, or keys command was not supported by twemproxy. I'm glad and thankful you took the lead with this.

[opuneet]

@gonzajg I've completed the changes to support micro batching in the Dyno connection pool.
And I've added support for mset, mget and msetnx. However, I see that mset and msetnx still need to be implemented on the Dynomite server side. Hence I'm going to refrain from merging my pull request into Dyno/master till we get the server side fix.

I'll file an issue on the server side, I've already spoken with @timiblossom about this.

Thanks.

[maksimlikharev]

any update on this one?
it looks like on the server side mget listed as supported, but driver has no implementation.

[jcacciatore]

@maksimlikharev I anticipate having this ready in the next couple of weeks.