Any way to make reshard process faster? It's really slow for use in production!

[Calabor-Hoc]

Reshard by redis-trib.rb migrating the keys in a rate of 60,000 keys per minutes. Although The resharding can be done on the fly not affecting incoming queries, it did affect multi-keys queries. In my business senario, I use a lot of lua script to wrap a series of operation involving multi-keys. Almost most of my business logic involve multi-keys operations. So, during a so-called "resharding in the runtime", my application donot have "runtime" at all... My application cannot do any thing when redis is resharding. And that is still good for me until I found this "down time" caused by reshard is so long that could consume a whole day. I think my data in redis isn't that big for production senario. So, I really think the reshard function provided by redis-trib.rb should be much much faster to have practical use.

I believe redis can do this much faster, after all it's just moving hundreds of MBs data over network cable line. I think the slow point may lie in somewhere in the redis-trib.rb. Did printing those dots for every key moved slow it down? Can sending those "migrate" commands in a pipeline rather then one by one improve a lot of speed?

[Calabor-Hoc]

My application use lua script to wrap a bunch of operations in one to achieve an atomic transaction effect, because my business really require a high-level data consistency . I really hope my app can benefit from the reshard feature

[HeartSaVioR]

@Calabor-Hoc
I modified redis-trib to use pipeline while moving keys in slot.
Please refer #2711 and check it out to see if it helps you.

[Calabor-Hoc]

@HeartSaVioR
We meet again!
Thank you a lot!