socket time out during replication #388

Closed
sima1 opened this Issue Mar 15, 2012 · 1 comment

Projects

None yet

3 participants

@sima1
sima1 commented Mar 15, 2012

Ver : 2.4.8

I've socket time out error just below during MR job using hadoop.

2012-03-15 10:31:11,284 WARN org.apache.hadoop.mapred.Child: Error running child
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.jedis.Protocol.process(Protocol.java:74)
at redis.clients.jedis.Protocol.read(Protocol.java:122)
at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:172)
at redis.clients.jedis.Connection.getBulkReply(Connection.java:161)
at redis.clients.jedis.Jedis.hget(Jedis.java:691)

so I thought it was jedis error or many connections occur it.

but I've finally found this issue isn't related jedis client only but also redis.
my redis server log as below at the same time with above hadoop job tracker log.

[22069] 15 Mar 10:31:07 - 22 clients connected (0 slaves), 2168951328 bytes in use
[22069] 15 Mar 10:31:08 * 10 changes in 300 seconds. Saving...
[22069] 15 Mar 10:31:08 * Background saving started by pid 27213
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Client closed connection
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:12 - Reading from client: Connection reset by peer
[22069] 15 Mar 10:31:16 - DB 0: 116 keys (0 volatile) in 128 slots HT.
[22069] 15 Mar 10:31:16 - 2 clients connected (0 slaves), 2168779304 bytes in use
[22069] 15 Mar 10:31:21 - DB 0: 115 keys (0 volatile) in 128 slots HT.
[22069] 15 Mar 10:31:21 - 2 clients connected (0 slaves), 2168779168 bytes in use
[27213] 15 Mar 10:31:22 * DB saved on disk
[22069] 15 Mar 10:31:22 * Background saving terminated with success

I have 30 clusters and I got this issue at 3 clusters and it has same log.

I also tracked #145.

please help me.

@sima1 sima1 closed this Mar 15, 2012
@pietern
pietern commented Mar 15, 2012

You found the issue, I assume?

@sima1 sima1 reopened this Mar 16, 2012
@mattsta mattsta closed this May 30, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment