dnsdist: Add server-side TCP Fast Open support

pdns/README-dnsdist.md

@@ -431,7 +431,7 @@ If you invoke setQR(1) on that, `dnsdist` knows you turned the packet into
 a response, and will send the answer directly to the original client.
 
 If you also called setTC(1), this will tell the remote client to move to
-TCP/IP, and in this way you can implement ANY-to-TCP even for downstream
+TCP, and in this way you can implement ANY-to-TCP even for downstream
 servers that lack this feature.
 
 Note that calling `addAnyTCRule()` achieves the same thing, without
@@ -469,7 +469,7 @@ addAction(MaxQPSIPRule(5), TCAction())
 ```
 
 This will respectively drop traffic exceeding that 5 QPS limit per IP or range, or return it with TC=1, forcing
-clients to fall back to TCP/IP.
+clients to fall back to TCP.
 
 To turn this per IP or range limit into a global limit, use NotRule(MaxQPSRule(5000)) instead of MaxQPSIPRule.
 
@@ -1142,8 +1142,8 @@ Here are all functions:
     * member `attachFilter(BPFFilter)`: attach a BPF Filter to this bind
     * member `toString()`: print the address this bind listens to
  * Network related:
-    * `addLocal(netmask, [true], [false])`: add to addresses we listen on. Second optional parameter sets TCP/IP or not. Third optional parameter sets SO_REUSEPORT when available.
-    * `setLocal(netmask, [true], [false])`: reset list of addresses we listen on to this address. Second optional parameter sets TCP/IP or not. Third optional parameter sets SO_REUSEPORT when available.
+    * `addLocal(netmask, [true], [false], [TCP Fast Open queue size])`: add to addresses we listen on. Second optional parameter sets TCP or not. Third optional parameter sets SO_REUSEPORT when available. Last parameter sets the TCP Fast Open queue size, enabling TCP Fast Open when available and the value is larger than 0.
+    * `setLocal(netmask, [true], [false], [TCP Fast Open queue size])`: reset list of addresses we listen on to this address. Second optional parameter sets TCP or not. Third optional parameter sets SO_REUSEPORT when available. Last parameter sets the TCP Fast Open queue size, enabling TCP Fast Open when available and the value is larger than 0.
  * Blocking related:
     * `addDomainBlock(domain)`: block queries within this domain
  * Carbon/Graphite/Metronome statistics related:
@@ -1239,7 +1239,7 @@ instantiate a server with additional parameters
     * `SkipCacheAction()`: don't lookup the cache for this query, don't store the answer
     * `SpoofAction(ip[, ip])` or `SpoofAction({ip, ip, ..}): forge a response with the specified IPv4 (for an A query) or IPv6 (for an AAAA). If you specify multiple addresses, all that match the query type (A, AAAA or ANY) will get spoofed in
     * `SpoofCNAMEAction(cname)`: forge a response with the specified CNAME value
-    * `TCAction()`: create answer to query with TC and RD bits set, to move to TCP/IP
+    * `TCAction()`: create answer to query with TC and RD bits set, to move to TCP
     * `TeeAction(remote[, addECS])`: send copy of query to remote, keep stats on responses. If `addECS` is set to `true`, EDNS Client Subnet information will be added to the query
  * Specialist rule generators
     * `addAnyTCRule()`: generate TC=1 answers to ANY queries received over UDP, moving them to TCP
@@ -1366,7 +1366,7 @@ instantiate a server with additional parameters
     * `setCacheCleaningDelay(n)`: set the interval in seconds between two runs of the cache cleaning algorithm, removing expired entries
     * `setStaleCacheEntriesTTL(n)`: allows using cache entries expired for at most `n` seconds when no backend available to answer for a query
  * DNSCrypt related:
-    * `addDNSCryptBind("127.0.0.1:8443", "provider name", "/path/to/resolver.cert", "/path/to/resolver.key", [false]):` listen to incoming DNSCrypt queries on 127.0.0.1 port 8443, with a provider name of "provider name", using a resolver certificate and associated key stored respectively in the `resolver.cert` and `resolver.key` files. The last optional parameter sets SO_REUSEPORT when available
+    * `addDNSCryptBind("127.0.0.1:8443", "provider name", "/path/to/resolver.cert", "/path/to/resolver.key", [false], [TCP Fast Open queue size]):` listen to incoming DNSCrypt queries on 127.0.0.1 port 8443, with a provider name of "provider name", using a resolver certificate and associated key stored respectively in the `resolver.cert` and `resolver.key` files. The fifth optional parameter sets SO_REUSEPORT when available. The last parameter sets the TCP Fast Open queue size, enabling TCP Fast Open when available and the value is larger than 0.
     * `generateDNSCryptProviderKeys("/path/to/providerPublic.key", "/path/to/providerPrivate.key"):` generate a new provider keypair
     * `generateDNSCryptCertificate("/path/to/providerPrivate.key", "/path/to/resolver.cert", "/path/to/resolver.key", serial, validFrom, validUntil):` generate a new resolver private key and related certificate, valid from the `validFrom` timestamp until the `validUntil` one, signed with the provider private key
     * `printDNSCryptProviderFingerprint("/path/to/providerPublic.key")`: display the fingerprint of the provided resolver public key

pdns/dnsdist-lua.cc

@@ -482,7 +482,7 @@ vector<std::function<void(void)>> setupLua(bool client, const std::string& confi
       g_ACL.modify([domain](NetmaskGroup& nmg) { nmg.addMask(domain); });
     });
 
-  g_lua.writeFunction("setLocal", [client](const std::string& addr, boost::optional<bool> doTCP, boost::optional<bool> reusePort) {
+  g_lua.writeFunction("setLocal", [client](const std::string& addr, boost::optional<bool> doTCP, boost::optional<bool> reusePort, boost::optional<int> tcpFastOpenQueueSize) {
       setLuaSideEffect();
       if(client)
 	return;
@@ -493,14 +493,14 @@ vector<std::function<void(void)>> setupLua(bool client, const std::string& confi
       try {
 	ComboAddress loc(addr, 53);
 	g_locals.clear();
-	g_locals.push_back(std::make_tuple(loc, doTCP ? *doTCP : true, reusePort ? *reusePort : false)); /// only works pre-startup, so no sync necessary
+	g_locals.push_back(std::make_tuple(loc, doTCP ? *doTCP : true, reusePort ? *reusePort : false, tcpFastOpenQueueSize ? *tcpFastOpenQueueSize : 0)); /// only works pre-startup, so no sync necessary
       }
       catch(std::exception& e) {
 	g_outputBuffer="Error: "+string(e.what())+"\n";
       }
     });
 
-  g_lua.writeFunction("addLocal", [client](const std::string& addr, boost::optional<bool> doTCP, boost::optional<bool> reusePort) {
+  g_lua.writeFunction("addLocal", [client](const std::string& addr, boost::optional<bool> doTCP, boost::optional<bool> reusePort, boost::optional<int> tcpFastOpenQueueSize) {
       setLuaSideEffect();
       if(client)
 	return;
@@ -510,7 +510,7 @@ vector<std::function<void(void)>> setupLua(bool client, const std::string& confi
       }
       try {
 	ComboAddress loc(addr, 53);
-	g_locals.push_back(std::make_tuple(loc, doTCP ? *doTCP : true, reusePort ? *reusePort : false)); /// only works pre-startup, so no sync necessary
+	g_locals.push_back(std::make_tuple(loc, doTCP ? *doTCP : true, reusePort ? *reusePort : false, tcpFastOpenQueueSize ? *tcpFastOpenQueueSize : 0)); /// only works pre-startup, so no sync necessary
       }
       catch(std::exception& e) {
 	g_outputBuffer="Error: "+string(e.what())+"\n";

pdns/dnsdist-lua2.cc

@@ -452,15 +452,15 @@ void moreLua(bool client)
       }
     });
 
-  g_lua.writeFunction("addDNSCryptBind", [](const std::string& addr, const std::string& providerName, const std::string& certFile, const std::string keyFile, boost::optional<bool> reusePort) {
+  g_lua.writeFunction("addDNSCryptBind", [](const std::string& addr, const std::string& providerName, const std::string& certFile, const std::string keyFile, boost::optional<bool> reusePort, boost::optional<int> tcpFastOpenQueueSize) {
       if (g_configurationDone) {
         g_outputBuffer="addDNSCryptBind cannot be used at runtime!\n";
         return;
       }
 #ifdef HAVE_DNSCRYPT
       try {
         DnsCryptContext ctx(providerName, certFile, keyFile);
-        g_dnsCryptLocals.push_back(std::make_tuple(ComboAddress(addr, 443), ctx, reusePort ? *reusePort : false));
+        g_dnsCryptLocals.push_back(std::make_tuple(ComboAddress(addr, 443), ctx, reusePort ? *reusePort : false, tcpFastOpenQueueSize ? *tcpFastOpenQueueSize : 0));
       }
       catch(std::exception& e) {
         errlog(e.what());

pdns/dnsdist.cc

@@ -73,9 +73,9 @@ bool g_syslog{true};
 
 GlobalStateHolder<NetmaskGroup> g_ACL;
 string g_outputBuffer;
-vector<std::tuple<ComboAddress, bool, bool>> g_locals;
+vector<std::tuple<ComboAddress, bool, bool, int>> g_locals;
 #ifdef HAVE_DNSCRYPT
-std::vector<std::tuple<ComboAddress,DnsCryptContext,bool>> g_dnsCryptLocals;
+std::vector<std::tuple<ComboAddress,DnsCryptContext,bool, int>> g_dnsCryptLocals;
 #endif
 #ifdef HAVE_EBPF
 shared_ptr<BPFFilter> g_defaultBPFFilter;
@@ -1636,11 +1636,11 @@ try
   if(g_cmdLine.locals.size()) {
     g_locals.clear();
     for(auto loc : g_cmdLine.locals)
-      g_locals.push_back(std::make_tuple(ComboAddress(loc, 53), true, false));
+      g_locals.push_back(std::make_tuple(ComboAddress(loc, 53), true, false, 0));
   }
 
   if(g_locals.empty())
-    g_locals.push_back(std::make_tuple(ComboAddress("127.0.0.1", 53), true, false));
+    g_locals.push_back(std::make_tuple(ComboAddress("127.0.0.1", 53), true, false, 0));
 
 
   g_configurationDone = true;
@@ -1667,11 +1667,13 @@ try
       setsockopt(cs->udpFD, IPPROTO_IPV6, IPV6_RECVPKTINFO, &one, sizeof(one));
 #endif
     }
-#ifdef SO_REUSEPORT
     if (std::get<2>(local)) {
+#ifdef SO_REUSEPORT
       SSetsockopt(cs->udpFD, SOL_SOCKET, SO_REUSEPORT, 1);
-    }
+#else
+      warnlog("SO_REUSEPORT has been configured on local address '%s' but is not supported", std::get<0>(local).toStringWithPort());
 #endif
+    }
 
 #ifdef HAVE_EBPF
     if (g_defaultBPFFilter) {
@@ -1700,10 +1702,18 @@ try
 #ifdef TCP_DEFER_ACCEPT
     SSetsockopt(cs->tcpFD, SOL_TCP,TCP_DEFER_ACCEPT, 1);
 #endif
+    if (std::get<3>(local) > 0) {
+#ifdef TCP_FASTOPEN
+      SSetsockopt(cs->tcpFD, SOL_TCP, TCP_FASTOPEN, std::get<3>(local));
+#else
+      warnlog("TCP Fast Open has been configured on local address '%s' but is not supported", std::get<0>(local).toStringWithPort());
+#endif
+    }
     if(cs->local.sin4.sin_family == AF_INET6) {
       SSetsockopt(cs->tcpFD, IPPROTO_IPV6, IPV6_V6ONLY, 1);
     }
 #ifdef SO_REUSEPORT
+    /* no need to warn again if configured but support is not available, we already did for UDP */
     if (std::get<2>(local)) {
       SSetsockopt(cs->tcpFD, SOL_SOCKET, SO_REUSEPORT, 1);
     }
@@ -1741,6 +1751,13 @@ try
       setsockopt(cs->udpFD, IPPROTO_IP, GEN_IP_PKTINFO, &one, sizeof(one));     // linux supports this, so why not - might fail on other systems
 #ifdef IPV6_RECVPKTINFO
       setsockopt(cs->udpFD, IPPROTO_IPV6, IPV6_RECVPKTINFO, &one, sizeof(one)); 
+#endif
+    }
+    if (std::get<2>(dcLocal)) {
+#ifdef SO_REUSEPORT
+      SSetsockopt(cs->udpFD, SOL_SOCKET, SO_REUSEPORT, 1);
+#else
+      warnlog("SO_REUSEPORT has been configured on local address '%s' but is not supported", std::get<0>(dcLocal).toStringWithPort());
 #endif
     }
 #ifdef HAVE_EBPF
@@ -1762,7 +1779,15 @@ try
 #ifdef TCP_DEFER_ACCEPT
     SSetsockopt(cs->tcpFD, SOL_TCP,TCP_DEFER_ACCEPT, 1);
 #endif
+    if (std::get<3>(dcLocal) > 0) {
+#ifdef TCP_FASTOPEN
+      SSetsockopt(cs->tcpFD, SOL_TCP, TCP_FASTOPEN, std::get<3>(dcLocal));
+#else
+      warnlog("TCP Fast Open has been configured on local address '%s' but is not supported", std::get<0>(dcLocal).toStringWithPort());
+#endif
+    }
 #ifdef SO_REUSEPORT
+    /* no need to warn again if configured but support is not available, we already did for UDP */
     if (std::get<2>(dcLocal)) {
       SSetsockopt(cs->tcpFD, SOL_SOCKET, SO_REUSEPORT, 1);
     }

pdns/dnsdist.hh

@@ -578,7 +578,7 @@ extern GlobalStateHolder<NetmaskGroup> g_ACL;
 
 extern ComboAddress g_serverControl; // not changed during runtime
 
-extern std::vector<std::tuple<ComboAddress, bool, bool>> g_locals; // not changed at runtime (we hope XXX)
+extern std::vector<std::tuple<ComboAddress, bool, bool, int>> g_locals; // not changed at runtime (we hope XXX)
 extern vector<ClientState*> g_frontends;
 extern std::string g_key; // in theory needs locking
 extern bool g_truncateTC;
@@ -642,7 +642,7 @@ bool fixUpResponse(char** response, uint16_t* responseLen, size_t* responseSize,
 void restoreFlags(struct dnsheader* dh, uint16_t origFlags);
 
 #ifdef HAVE_DNSCRYPT
-extern std::vector<std::tuple<ComboAddress,DnsCryptContext,bool>> g_dnsCryptLocals;
+extern std::vector<std::tuple<ComboAddress,DnsCryptContext,bool,int>> g_dnsCryptLocals;
 
 int handleDnsCryptQuery(DnsCryptContext* ctx, char* packet, uint16_t len, std::shared_ptr<DnsCryptQuery>& query, uint16_t* decryptedQueryLen, bool tcp, std::vector<uint8_t>& reponse);
 bool encryptResponse(char* response, uint16_t* responseLen, size_t responseSize, bool tcp, std::shared_ptr<DnsCryptQuery> dnsCryptQuery);