Redis Cluster Error - Error processing response from Redis node!

[kushalkh]

Expected behaviour

try{
$data = $redisCluster->get("some_key");
}
catch(RedisClusterException $e){
echo $e->getMessage();
}

This should work always and it does most of the times except the intermittent issue.

Actual behaviour

Intermittent issue on Redis Cluster. The RedisClusterException message says "Error Message: Error processing response from Redis node!" on "get" command.

We're running 36 masters with 1 slave per master. Slaves are set to be used for read only if the master is unreachable.

Read and write timeouts are set to 30 sec each.

I'm seeing this behaviour on

• OS: Linux (CentOS 7)
• Redis: 5.0.5
• PHP: 7.0.33
• phpredis: 5.0.2

Steps to reproduce, backtrace or example script

Very intermittent. No guaranteed way of reproducing and it happens on random keys without any specific pattern. All instances are always up and running.

I've checked

• [x ] There is no similar issue from other users
• Issue isn't fixed in develop branch

Please let me know if there's any way to check the specific reason for this error. I believe it can return this error in multiple scenarios.
One interesting thing is, before we increased read and write timeouts from 1 sec to 30 sec, we were getting similar intermittent "Timed out finding data in node" error.

Is this error also network related? Or can it be something else? What's the best way to figure it out?

Thanks for the help.

[guyromb]

I am experiencing the same issue.

[dreamMonkeyZ]

also me,

[kushalkh]

Increasing the timeouts to a relatively large number like 20 seconds and choosing the setting to read from slaves when master isn’t available worked for me

[alemferreira]

+1

However we're getting this error on setex, not on get

[michael-grunder]

Hi,

The two errors mentioned in this thread are related.

1. Timed out attempting to find data in the correct node! error is a timeout.
2. Error processing response from Redis node! is a network failure.

PhpRedis has a command loop where we send a command and then process server responses until (a) we have the proper response, (b) some number of milliseconds has elapsed, or (c) a network error occurred reading a response from one of the servers.

Keep in mind that Redis cluster may reply MOVED or ASKING if slots are resharding, etc, which is why the commands are handled in a loop.

I think it may be possible to improve the command loop and retry some commands after a network failure (at least until the timeout elapses) but we're probably limited to doing this for read-only commands.

As others have mentioned you can likely alleviate the issue by distributing reads across replicas and bumping timeouts.

[mrAndersen]

Same issue here. Are there any chances to get more verbose output for that error?
Because it happens at random, 3 masters, 3 slaves, all of them are added in cluster initialiaztion

$this->cluster = [
//3 masters,
//3 slaves here
]
$this->redis = new RedisCluster(null, $this->cluster, 20, 20, true, $this->clusterPassword);