dnsdist: Implement TCP out-of-order

[rgacogne]

Short description

This PR implements TCP out-of-order support for dnsdist:

• for incoming queries (between the client and dnsdist) ;
• as well as for outgoing queries (between dnsdist and the backend).

We try to reuse the existing TCP connections as much as possible, but we might need to open many connections to the same backend, especially if that backend does not support out-of-order. We also don't mix queries from different incoming TCP connections on the same TCP connection to the backend, since that would require taking care of query ID collisions, which we currently detect. An existing TCP connection to the backend can be re-used for a different incoming connection later, though.

This PR break the TCP handling code in two parts, one handling incoming TCP connections and a second one handling outgoing ones.

It still needs a serious review and more testing.

DNS Shotgun seems to be quite happier with it, by the way:

Checklist

I have:

• read the CONTRIBUTING.md document
• compiled this code
• tested this code
• included documentation (including possible behaviour changes)
• documented the code
• added or modified regression test(s)
• added or modified unit test(s)

[paddg]

How many TCP connections did you simulate during your 6kqps test?

[rgacogne]

Roughly 10k:

[paddg]

Very good! Did you test with real DNS queries or with synthetic? If I test with real DNS traffic, the response rate is below 60%

[rgacogne]

Did you test with real DNS queries or with synthetic? If I test with real DNS traffic, the response rate is below 60%

Yes, that test is done with the data set mentioned in #9572, which is real-word traffic. Note that the test is done over DNS over TLS, and I tuned a few things:

• setTCPRecvTimeout(30): read time out set to 30s ;
• setMaxTCPClientThreads(8): 8 TCP worker threads ;
• 2 * addTLSLocal("[::1]:8443", 'server.pem', 'server.key', { reusePort=true}, numberOfStoredSessions=0, maxInFlight=5 }): 2 threads doing accept() ;
• 8 * newServer({ address = "[::1]", maxInFlight=10}): 8 backend threads ;

On the recursor side, both #9495 and #9572 are applied, and I have:

• client-tcp-timeout=30 ;
• dnssec=validate ;
• max-mthreads=8192 ;
• max-tcp-clients=500000 ;
• pdns-distributes-queries=no ;
• qname-minimization=yes ;
• reuseport=yes ;
• threads=8.

[paddg]

* numberOfStoredSessions=0

Is this meant as unlimited? Because "Setting this value to 0 disables stored session entirely"

[rgacogne]

Is this meant as unlimited? Because "Setting this value to 0 disables stored session entirely"

No, it is intended to disable sessions stored server-side, since client-side tickets are much nicer to the server and more realistic in anycast setups anyway, since the session-ticket key can be shared between servers much more easily than the server-side sessions.

[rgacogne]

Squashed to clean up the history a bit, and to make sure all commits build and pass the unit tests.

[omoerbeek]

It's quite a big PR, so I won't state I looked at every line of code. But what I checked looks good and basic testing show it works as advertised once I git the settings right.

related to that: please document that for it to work you should set both incoming and outgoing maxInFlight.

[rgacogne]

It's quite a big PR, so I won't state I looked at every line of code. But what I checked looks good and basic testing show it works as advertised once I git the settings right.

I did not expect you to look at every single detail, don't worry! :-) Thanks a lot for the review and tests!

related to that: please document that for it to work you should set both incoming and outgoing maxInFlight.

I just added more words on how that feature works, that was clearly lacking!