Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cluster mode #302

Merged
merged 3 commits into from Jun 16, 2021
Merged

Support cluster mode #302

merged 3 commits into from Jun 16, 2021

Conversation

ShooterIT
Copy link
Member

@ShooterIT ShooterIT commented Jun 8, 2021

resolves #219

Deploy Kvrocks Cluster

In our cluster solution, you should firstly deploy some kvrocks instances and set replication relationship, and
Kvrocks doesn't support to communicate with each others, so we need to tell every node the cluster topology.
The command is CLUSTER SETNODES

  • CLUSTERX SETNODES $ALL_NODES_INFO $VERSION $FORCE
    $ALL_NODES_INFO format: $node_id $ip $port $role $master_node_id $slot_range. $role: master or slave, $master_node_id: master_node_id if it is slave, otherwise, this value is -. slave needn't set $slot_range.
    Example:
    07c37dfeb235213a872192d90877d0cd55635b91 127.0.0.1 30004 slave e7d1eecce10fd6bb5eb35b9f99a514335d9ba9ca
    67ed2db8d677e59ec4a4cefb06858cf2a1a89fa1 127.0.0.1 30002 master - 5461-10922
    
    $VERSION means current cluster topology version that is stored or generated by management component such as config server or meta-server.

Kvrocks can know itself node id by compared with itself ip and port in received all nodes info, but you maybe bind kvrorks to 0.0.0.0, so it is necessary to set NODEID explicitly to let it know itself. As the same with redis cluster, node id is 40 characters length.

  • CLUSTERX SETNODEID $NODE_ID

Moreover, cluster doesn't handle failover and migration, you should set cluster topology again after failover, of course, you should do that if you add some new nodes into cluster.

Access Kvrocks Cluster

After setting cluster topology, we have tested, it adopt redis-cli, redis-benchmark, python redis cluster sdk, and redis-cluster-proxy. So you may doesn't need to change your redis cluster request code to access kvrocks cluster.

Have a try

In tool/try_cluster, you can change BIN_PATH of kvrocks bin path, and then
./try_cluster.sh start to create a small cluster that has 6 nodes(3 maters and 3 replicas)

then run ./try_cluster.sh watch to observe cluster nodes, you also can do some tests by
redis-cli, redis-benchmark, redis cluster sdk and redis-cluster-proxy for your fun.

finally you can run ./try_cluster.sh stop to stop all nodes and clean up directory.

Supplements

Use CLUSTERX command to set cluster topology instead of CLUSTER #324

Copy link
Member

@git-hulk git-hulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, the cluster topology and version would store on local storage?

@ShooterIT
Copy link
Member Author

@git-hulk That is a good question, I still consider that, we must should know the nodes may keep old cluster topology if restarted. That may cause terrible things if the IP and port are reused.

@ShooterIT ShooterIT added major decision Requires project management committee consensus release notes labels Jun 14, 2021
@git-hulk
Copy link
Member

yup, use ip:port as the unique identifier would cause problems, the node id would be better if we also persist it.

@ShooterIT ShooterIT merged commit fd0a0af into apache:unstable Jun 16, 2021
@ShooterIT ShooterIT deleted the cluster-mode branch June 16, 2021 13:00
ShooterIT added a commit that referenced this pull request Jun 16, 2021
One cluster mode is similar with redis cluster, please see #219
ShooterIT added a commit to ShooterIT/kvrocks that referenced this pull request Jul 15, 2021
One cluster mode is similar with redis cluster, please see apache#219
@ShooterIT ShooterIT mentioned this pull request Jul 15, 2021
ShooterIT added a commit that referenced this pull request Jul 19, 2021
One cluster mode is similar with redis cluster, please see #219
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
major decision Requires project management committee consensus release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[NEW] Support redis cluster mode
3 participants