Redis 7.0 cluster access fails with an error: got 4 elements in cluster info address, expected 2 or 3

[vamsidarbhamulla]

Brief summary

When trying to pass in multiple node names as strings for redis 7.0 addrs option property we receive back an error mentioning as below

ERRO[0000] Uncaught (in promise) got 4 elements in cluster info address, expected 2 or 3 .

Looks like go-redis has documented this observation and suggested to use go-redis v9 as a comment in one of their issues:
redis/go-redis#2085 (comment)

k6 version

k6 v0.43.1 ((devel), go1.20.1, )

OS

mac os 12 , darwin/arm64

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Simple k6 script to interact with redis 7.0 cluster :

import redis from 'k6/experimental/redis';

const client = new redis.Client({
  addrs: ['localhost:53219', 'localhost:53220', 'localhost:53221'],
});

export default function() {
  const key = 'mykey';
  const value = 'myvalue';

  client.set(key, value);
  const result = client.get(key);

  console.log(result);
}

Example docker-compose.yml for redis cluster 7.0:

version: '3.8'

services:
  # Here we have six Redis containers with Cluster mode enabled,
  # three of them will work as master nodes and each one of
  # will have a replica, so in case of failures, the replica becomes the master.
  # They are configured by the `cluster_initiator` container.
  redis_1:
    image: 'redis:7.0.7'
    container_name: redis_1
    ports:
      - "53219:6379"
    volumes:
      - redis_1_data:/data
    command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.2

  redis_2:
    image: 'redis:7.0.7'
    container_name: redis_2
    ports:
      - "53220:6379"
    volumes:
      - redis_2_data:/data
    command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.3

  redis_3:
    image: 'redis:7.0.7'
    container_name: redis_3
    ports:
      - "53221:6379"
    volumes:
      - redis_3_data:/data
    command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.4
  
  redis_4:
    image: 'redis:7.0.7'
    container_name: redis_4
    ports:
      - "6379"
    volumes:
      - redis_4_data:/data
    command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.5

  redis_5:
    image: 'redis:7.0.7'
    container_name: redis_5
    ports:
      - "6379"
    volumes:
      - redis_5_data:/data
    command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.6
  
  redis_6:
    image: 'redis:7.0.7'
    container_name: redis_6
    ports:
      - "6379"
    volumes:
      - redis_6_data:/data
    command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.7

  # Ephemeral container to create the Redis cluster connections.
  # Once the setup is done, this container shuts down
  cluster_initiator:
    image: 'redis:7.0.7'
    container_name: cluster_initiator
    tty: true
    depends_on:
      - redis_1
      - redis_2
      - redis_3
      - redis_4
      - redis_5
      - redis_6
    command: bash -c "sleep 5 && echo 'yes' | redis-cli --cluster create \
        173.18.0.2:6379 \
        173.18.0.3:6379 \
        173.18.0.4:6379 \
        173.18.0.5:6379 \
        173.18.0.6:6379 \
        173.18.0.7:6379 \
        --cluster-replicas 1
      echo '🚀 Redis cluster ready.' "
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.8

  # Web UI to browse through our Redis data across all nodes
  redis_commander:
    image: rediscommander/redis-commander:latest
    container_name: redis_web
    environment:
      REDIS_HOSTS: "local:redis_1:6379,local:redis_2:6379,local:redis_3:6379"
    ports:
      - "5051:8081"
    depends_on:
      - redis_1
      - redis_2
      - redis_3
      - redis_4
      - redis_5
      - redis_6
      - cluster_initiator
    networks:
      redis_cluster_net:
        ipv4_address: 173.18.0.9

volumes:
  redis_1_data: {}
  redis_2_data: {}
  redis_3_data: {}
  redis_4_data: {}
  redis_5_data: {}
  redis_6_data: {}

# Rename the default network so we can easily identify it
# Across all containers
networks:
  redis_cluster_net:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 173.18.0.0/16

Expected behaviour

k6 redis script should be able to interact with redis 7.0 cluster.

Actual behaviour

ERRO[0000] Uncaught (in promise) got 4 elements in cluster info address, expected 2 or 3

Specific issues identified related to the same:

Might need to use go-redis v9 :

redis/go-redis#2082
redis/redis#9530

[oleiade]

Hi @vamsidarbhamulla 👋🏻

thanks for reporting it! We're not users of the cluster mode internally so it's good to get feedback on that feature specifically. I really appreciate that you even offered a docker-compose file to reproduce the issue. I'll give it a spin, and see if upgrading to go-redis v9 does solve this issue.

If that's the case we'll look into upgrading the dependency indeed 👍🏻

[oleiade]

I was able to reproduce the issue indeed 👍🏻

I've put together a branch of xk6-redis with go-redis updated to version 9, and it seems to sort the issue out. If you could give it a try and build a k6 executable, using xk6, including that xk6-redis branch, that would be much appreciated 🙇🏻

go install go.k6.io/xk6/cmd/xk6@latest
xk6 build v0.43.1 --with github.com/grafana/xk6-redis=.
./k6 run cluster.js

I've included a docker-compose setup similar to yours in the branch to spin up a cluster, but ran into weird issues on MacOS where nodes can't talk to each other (it seems to work on linux). Thus, if you could take it for a spin to verify that it addresses your use case, that would raise our level of confidence that we're going in the right direction 👍🏻

Besides that, once we update the dependency some of our tests break, and some API changes have been made to the options, and I believe we'd need to put a bit more work into the changes to be able to deliver it to the k6/experimental/redis module. That would push the fix to either the next version of k6 (ETA release around mid-june), or even maybe the next one.

[vamsidarbhamulla]

Hi @oleiade ,

Thanks for looking into it and providing a working example using the xk6-redis repo branch. Before raising this bug, I did try to verify redis 7.0 cluster connection using go-redis v9 and it did work on local branch changes.

But after looking into k6/experimental/redis code changes in this repo with v8 version I felt it would be better to get this reported as it looked like a major change.

I can confirm the xk6-redis branch is working with redis 7.0 cluster, I will see if I can create a local build with these changes for my use-case till the next version of k6 releases.

[oleiade]

Hey @vamsidarbhamulla

Thanks a lot for looking into it and giving it a try. Let me know if you need any support in producing a build with the fix 👍🏻

I'll take a look at why the tests break currently when I upgrade the redis dependency. Depending on the complexity of the problem, I might be able to get the updated dependency merged for the next version of k6 indeed, but not promises. I'll keep you updated as I make progress 🙇🏻‍♂️

[vamsidarbhamulla]

Hi @oleiade - Thanks again. If it is possible I would request to get an xk6-redis build version with this fix for the cluster connection logic.

I can directly build a local k6 executable if this change is available as a specific version.

[oleiade]

Hey @vamsidarbhamulla 👋🏻

We have recently updated the go-redis dependency of xk6-redis, updated how the Client is constructed to cater to more complex cases such as clustering, and generally improved support for TLS. Those changes will be reflect in the k6/experiemental/redis module with the upcoming version v0.48, but are also already available within the master branch of xk6-redis.

If you have the opportunity, could you take it for a spin and let us know if your issue is addressed?

Thank you 🙇🏻