Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
redis-trib: use pipeline to speed up moving slot #2711
Test environment is below,
Test result is here,
b) verbose = false
Increasing fetch size of keysinslot and using pipeline seems to be 2 ~ 2.5 times faster than current.
Btw, just modifying verbose option to 'false' also reduces more than 25 secs.
referenced this pull request
Aug 3, 2015
Please note that it is just a benchmark, but I also expect more speedup than 2x.
ps. Maybe we need to increase bulk size. (10000 could be small.)
The reason why the speedup is not the one expected is the way MIGRATE is implemented. It requires to chat with other nodes and get a reply. So instead of implementing pipelining, if we truly want much faster migration, we should pipeline MIGRATE itself internally. I'm trying to do this right now for 3.2 and possibly will back port to 3.0. Migrate in this use case will have a new option where it is possible to specify multiple keys, so that the source node will send all the keys and will later retrieve all the replies, making the process much faster. News soon, on it.