net/tcp: Add sockopt to allow TIME_WAIT reuse

[dhobsd]

When both the requesting socket and the socket found in TIME_WAIT state
are requesting to reuse TIME_WAIT-state sockets, allow us to find the TW
socket as unique.

Previously, this was gated on a global sysctl, which is not ideal for our
use case.

[dormando]

How does this not break NAT in the same way tw_reuse always does?

[dhobsd]

It doesn't, unless you only use it as a client, in which case it's probably always fine. The intent of the sockopt is for us to only be using it as a client, when we connect out to origins; @crucially seemed to think this would be OK behavior.

[dhobsd]

Oh, it will also only reuse the socket if both the discovered tw sock and the connecting sock agree for reuse. So we won't reuse a socket that is in TW state that happens to belong to some connection where this would be problematic, or when our connecting socket belongs to an application that doesn't want to do this. Since both sockets had to request this behavior when they were created, this differs from tw_reuse in that it is not global and will not have any impact on applications that do not want to risk it.

I do think that our very low tw timer value (5 seconds) is low enough that we would probably already break old NAT gateways where there's enough loss that they'd be retransmitting; 5 seconds is way under what such a gateway would consider 2*MSL (which is probably always going to be 120s).

So those are the reasons I think it's a safe change.