need to cleanup sockets for clients do not end cleanly.

[troy2914]

We had the situation where 68 workers were working on one ip. Looking at netstat
it revealed that all 68 to that ip where in the state FIN_WAIT2. To which the client should
have responded with FIN,ACK, but apparently did not.

Expected behaviour
after 5 minutes in FIN_WAIT2 the socket should be force-ably cleaned up.

IRRd version you are running
IRRd -- version 4.1.8

[mxsasha]

Sounds good, will dig into it.

[mxsasha]

@troy2914 do you have some idea of how long these sockets were lingering? Are we talking an accumulation to 68 over a time frame of hours? More? A minute?

[mxsasha]

Initial thoughts: the close was initiated from IRRD, which is normal, but the remote end did not finish the closing of the socket, making the IRRD end stuck in FIN_WAIT2 (state diagram), waiting for a FIN from the remote end. It is possible that this line (docs for close(), docs for shutdown()) blocks at this point. If the IRRD server process is blocked, the socket is still attached, preventing the kernel from cleaning it up even after tcp_fin_timeout.

Possible solution: set a timeout on the socket, which apparently we don't do at all in the whois TCP server. Would need a bit of testing though.

[mxsasha]

I haven't really been able to reproduce this, so I'm planning to move forward with settimeout on the socket. It definitely seems to need a highly rare kind of misbehaving client/OS/network.