add support for cluster mode #42
Conversation
|
|
||
| // store key for other connection, if queue is not full | ||
| key_index_pool* key_idx_pool = m_key_index_pools[m_slot_to_shard[hslot]]; | ||
| if (key_idx_pool->size() < KEY_INDEX_QUEUE_MAX_SIZE) { |
There was a problem hiding this comment.
Did you confirm that in all possible scenarios it is ok for the obj_gen to return a key that is simply discarded (because the queue is full)?
There was a problem hiding this comment.
The answer divide for two parts:
-
For Random/Gaussian:
We think that it is ok for those options, since the discarding of keys suppose to be relative to the selected pattern, so in the overall the distribution will stay as it suppose to be (Random/Gaussian). -
For Sequential:
Here we have problem that we know about it...
The issue start when the different connections are not symmetrical (networking/shards conuts ...).
In that case one connection can be slow from other and his queue starting to grow over and over and the cluster-client start to discard keys.
In sequential distribution we have issue if user ask for keys (for ex) between 1 - 1000000.
If user set memtier to --test-time and not --requests option, there isn't really issue.
Since the connections are independent, it could be gaps in the DB even if cluster-client will not discard any keys.
The problem is only when user set memtier to --requests option, and if he ask 1000000 request in range of 1 - 1000000 he expect that at the end of the test, the DB will be with those exactly keys (1-10000000), and since we discards keys this will not be the result.
We think about two solutions for this case:
- add option for user to set the queue limit size.
- add option to do some balancing between the connections, so if some connection reach to some threshold, the other connection start to send less requests or stop send request, until the slower connection go down below the the threshold.
What do you think?
Is this issue is critical? or we can solved in the future.
The solutions are good, do you have any other idea?
There was a problem hiding this comment.
I don't think it's critical, it should probably be documented though in a section about cluster support caveats.
cluster_client.cpp
Outdated
|
|
||
| /////////////////////////////////////////////////////////////////////////////////////////////////////// | ||
|
|
||
| cluster_client::cluster_client(client_group* group) : client(group)/*, m_finished_set(false), m_reqs_generated(0)*/ |
There was a problem hiding this comment.
Make sure unused code is removed, not just commented
There was a problem hiding this comment.
ok, I will remove it
YaacovHazan
force-pushed
the
support_cluster_mode
branch
from
449bf30 to
c82ffb5
Compare
new 'cluster_client' add to support cluster mode. the new client use CLUSTER_SLOTS command to retrive the key-->node mapping, create separate connection fot each node, and send the commands to the appropriate connection according to hash calculation.
YaacovHazan
force-pushed
the
support_cluster_mode
branch
from
c82ffb5 to
528d3bc
Compare
3 participants
new 'cluster_client' add to support cluster mode.
the new client use CLUSTER_SLOTS command to retrive the
key-->node mapping, create separate connection fot each node,
and send the commands to the appropriate connection according
to hash calculation.