Configuring Idle connection time CPP2 ThriftServer

[varunsharmagit]

We are running into the following issue. We have a proxy (CPP2 server + client) which manages connections to a bunch of backend servers - the backend servers are CPP2 ThriftServer. We see that if the connection remains idle for a while (no requests) - the connect is closed and we suspect this is because the CPP2 ThriftServer closes connections idle for a certain period of time. We start seeing a large number of "Channel is !good inside our proxy at this point". We have two options:

1. Increasing this time to infinity - this would make our proxy control connections completely and since the stack is all CPP2, we can get away with very few connections either ways

2. Doing more connection management in the proxy which would be painful. Is it possible to perform a "reconnect" on a TAsyncSocket which is no longer "good()" ?

[varunsharmagit]

We did verify closed connections by running netstat and finding the connections moving from ESTABLISHED to CLOSE_WAIT state.

[djwatson]

server->setIdleTimeout(0)

^ this turns off the idle connection closing. At FB we do use smarter clients that automatically reconnect though, but we haven't open sourced that code.

HeaderClientChannel has a setCloseCallback() that you could use to implement that functionality, so you'd get an async notification that the server closed the client connection in the proxy

[varunsharmagit]

Thanks. Does the HeaderClientChannel's closeCallback get called when the backend server crashes/restarts - I believe that would also void the connection ?

[djwatson]

It fires on tcp close or tcp reset events, it doesn't do any active keepalive.