Skip to content

hector lost downed host #439

Closed
daningaddr opened this Issue Mar 15, 2012 · 3 comments

3 participants

@daningaddr

If there is exception in addCassandraHost() before adding host to hostPools, since addCassandraHost does not throw exception, the host will be removed from downedHostQueue, and the host will get lost.

if(downedHostQueue.contains(cassandraHost) && verifyConnection(cassandraHost)) { connectionManager.addCassandraHost(cassandraHost);
downedHostQueue.remove(cassandraHost);
return;
}

Here is the exception I got

2012-03-15 05:39:20,944 [Hector.me.prettyprint.cassandra.connection.CassandraHostRetryService-1] Transport exception host to HConnectionManager: s2.dsat4.netseer.com(10.210.101.116):9160
me.prettyprint.hector.api.exceptions.HectorTransportException: Unable to open transport to s2.dsat4.netseer.com(10.210.101.116):9160 , java.net.SocketTimeoutException: connect timed out
at me.prettyprint.cassandra.connection.client.HThriftClient.open(HThriftClient.java:144)
at me.prettyprint.cassandra.connection.client.HThriftClient.open(HThriftClient.java:26)
at me.prettyprint.cassandra.connection.ConcurrentHClientPool.createClient(ConcurrentHClientPool.java:147)
at me.prettyprint.cassandra.connection.ConcurrentHClientPool.(ConcurrentHClientPool.java:53)
at me.prettyprint.cassandra.connection.LeastActiveBalancingPolicy.createConnection(LeastActiveBalancingPolicy.java:59)
at me.prettyprint.cassandra.connection.HConnectionManager.addCassandraHost(HConnectionManager.java:116)
at me.prettyprint.cassandra.connection.CassandraHostRetryService$1.run(CassandraHostRetryService.java:75)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: connect timed out
at org.apache.thrift.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
at me.prettyprint.cassandra.connection.client.HThriftClient.open(HThriftClient.java:138)
... 14 more
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.apache.thrift.transport.TSocket.open(TSocket.java:17

@patricioe
Collaborator

it should be:

 if(downedHostQueue.contains(cassandraHost) && verifyConnection(cassandraHost)) {   
      if(connectionManager.addCassandraHost(cassandraHost)) 
           downedHostQueue.remove(cassandraHost); 
      return;
 }
@patricioe patricioe closed this Mar 17, 2012
@hadolphs
hadolphs commented Apr 2, 2012

guys, did this fix and other connection pool fixes make it to the 0-8 branch?

@patricioe
Collaborator

Not yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.