TCP 3 way handshake

[sigmonsays]

How do you work around connection resets during a 3 way handshake if the number of listeners changes?

[jbenet]

What do you mean by "if the number of listeners changes"? as in having multiple listeners for concurrency? (and IIRC, this can happen across multiple processes when both SO_REUSEPORT and SO_REUSEADDR are set).

i suspect it's the kernel's job, but maybe useful to give a clear example of where the problem may emerge.

[sigmonsays]

Tcp resets are sent for any connection in the middle of a 3way handshake. It is a known kernel issue but defeats usefulness of this library. lie the load balancing use case, graceful restart, etc.

It really should be made clear upfront. Even your example is not going to work as expected.

Details are explained at https://lwn.net/Articles/542629/

[jbenet]

Are you talking about this?

If the number of listening sockets bound to a port changes because new servers are started or existing servers terminate, it is possible that incoming connections can be dropped during the three-way handshake. The problem is that connection requests are tied to a specific listening socket when the initial SYN packet is received during the handshake. If the number of servers bound to the port changes, then the SO_REUSEPORT logic might not route the final ACK of the handshake to the correct listening socket. In this case, the client connection will be reset, and the server is left with an orphaned request structure. A solution to the problem is still being worked on, and may consist of implementing a connection request table that can be shared among multiple listening sockets.

• This is only for connections caught in the middle of a 3-way handshake during a listener change. i think it's only likely to affect servers receiving hundreds of new connections per second. high perf servers may want to modify this pkg to use epoll and kqueue better, as today this pkg wastes 1 fd per connection.
• This was 2 years ago, hopefully someone fixed it by now. How would we know? is there a thread somewhere? could ask Michael.
• Listening from multiple listeners isn't my use case-- my use case is dialing + listening from the same socket, and this pkg achieves that.

[anacrolix]

• Listening from multiple listeners isn't my use case-- my use case is dialing + listening from the same socket, and this pkg achieves that.

Given this point, the lack of follow-up reports, and any ongoing conversation on this point, I think it's worth closing this issue due to age. Any deviation in behaviour from stock Go will be ameliorated by a coming PR for this package possible since golang/go#9661 has been fixed. Cheers