Principle of Redis Cluster

Architecture

Structure interpretation:

• The entire deployment conforms to the Redis Cluster cluster topology and introduces Predixy as the agent layer of the cluster.
• The group of Redis Cluster corresponds to a statfulset, such as XX-0, then the active and standby in the group will be named XX-0-0 and XX-0-1, and the status in the CR object will dynamically record the slot corresponding to the group Bit.
• The underlying storage of the Redis instance is optional. Combined with the storageclass, it can be connected to the local disk (empty dir) or ceph (pvc), etc.
• Each Redis Pod has a sidecar exporter mirror, which can smoothly connect to prometheus.

Redis Cluster CRD example:

apiVersion: cache.tongdun.net/v1alpha1
  kind: RedisCluster
  metadata:
    name: redis-cluster-rat
    namespace: redis-cluster
  spec:
    app: cluster-rat
    capacity: 32768
    dc: hz
    env: production
    image: tongduncloud/redis-cluster:0.2
    monitorimage: tongduncloud/redis-exporter:1.0
    netmode: ClusterIP
    proxyimage: tongduncloud/predixy:1.0
    proxysecret: abc
    realname: dba
    secret: abc
    size: 3
    storageclass: ""
    vip: ""

Cluster creation

Operator is responsible for the organization process of the cluster, and the entire process is to code the operation and maintenance deployment process.

Here are key steps:

• The foreground service calculates the number of groups and the maxmemroy of a single node based on the storage size entered by the user.
• In the process of creating sts according to the number of groups, it will wait for all (number of groups*2) Redis Pods to provide services normally. At this time, the Pods are independent and have no roles. At this time, call the cluster to create the API command to organize XN-0 to become Cluster.
• X-N-1 is added to the corresponding group in the cluster as a slave.

Cluster expansion

Cluster expansion is a complicated process. On the one hand, expansion is divided into horizontal expansion and vertical expansion. On the other hand, how to minimize the impact of online expansion on the line also needs to be considered.

Vertical expansion

• Vertical expansion has the smallest impact on online services. Only if the storage demand size does not exceed the current number of groups * 32G (single node storage upper limit), vertical expansion will be implemented.
• As long as it is recognized as a vertical expansion requirement by the Operator, the new capacity of a single node is calculated, and the persistence will be updated through config set and configmap.

Horizontal expansion

• When the storage seeking exceeds the current number of groups*32G, it will enter the horizontal expansion logic.
• Calculate the number of groups that need to be added, create the corresponding statefulset, and join the cluster in turn, and achieve data balance through Rebalance.
• After the data is balanced, the new slot information will be recorded in the status of the CR object.

Failover

Processing flow:

• When a single point of failure occurs, the failed node will be identified and deleted.
• For the newly created Redis instance in the group, the group corresponding to the corresponding sts will be identified and the master-slave synchronization will be established with the current master of the corresponding group.