Dnsdist doh [DO NOT MERGE] #6911

ahupowerdns · 2018-08-31T16:08:25Z

Short description

This PR adds DNS over HTTPS support to dnsdist, based on libh2o.

Checklist

I have:

read the CONTRIBUTING.md document
compiled this code
tested this code
included documentation (including possible behaviour changes)
documented the code
added or modified regression test(s)
added or modified unit test(s)
checked that this code was merged to master

johndickinson · 2018-09-06T09:17:46Z

I had to upgrade to bionic (from trusty) to get libh2o-evloop and also had to specify LIBS=-lwslay to fix link errors like

/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libh2o-evloop.so: undefined reference to `wslay_event_want_read’

The full configure line I used was ./configure --enable-dns-over-tls --enable-dns-over-https --with-libcrypto=/opt/openssl-1.1.1-pre9 --enable-re2 CXXFLAGS=-I/opt/openssl-1.1.1-pre9/include CPPFLAGS=-I/opt/openssl-1.1.1-pre9/include LDFLAGS=-L/opt/openssl-1.1.1-pre9/lib LIBS=-lwslay

pieterlexis · 2018-09-06T09:28:10Z

@johndickinson,

What does config.log say about libh2o? And can you share the libh2o-evloop.pc? On arch linux, the -lwslay is not needed (and not in the .pc file). I think this is because the arch PKGBUILD installs the library as a .a and Debian/ubuntu as a .so.

johndickinson · 2018-09-06T09:44:25Z

config.log:
...
configure:20525: checking for LIBH2OEVLOOP
configure:20532: $PKG_CONFIG --exists --print-errors "libh2o-evloop"
configure:20535: $? = 0
configure:20549: $PKG_CONFIG --exists --print-errors "libh2o-evloop"
configure:20552: $? = 0
configure:20590: result: yes
...
ac_cv_env_LIBH2OEVLOOP_CFLAGS_set=
ac_cv_env_LIBH2OEVLOOP_CFLAGS_value=
ac_cv_env_LIBH2OEVLOOP_LIBS_set=
ac_cv_env_LIBH2OEVLOOP_LIBS_value=
...
pkg_cv_LIBH2OEVLOOP_CFLAGS=
pkg_cv_LIBH2OEVLOOP_LIBS=-lh2o-evloop
...
HAVE_LIBH2OEVLOOP_FALSE='#'
HAVE_LIBH2OEVLOOP_TRUE=''
...
LIBH2OEVLOOP_CFLAGS=''
LIBH2OEVLOOP_LIBS='-lh2o-evloop'
...
#define HAVE_LIBH2OEVLOOP 1

$ cat /usr/lib/x86_64-linux-gnu/pkgconfig/libh2o-evloop.pc
prefix=/usr
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=/usr/lib/x86_64-linux-gnu

Name: libh2o-evloop
Description: An optimized HTTP/1.x & HTTP/2 library
URL: https://h2o.examp1e.net/
Version: 0.13.4
Libs: -L${libdir} -lh2o-evloop
Cflags: -I${includedir}

zeha · 2018-09-06T11:48:22Z

Rgd -lwslay: https://sources.debian.org/src/h2o/2.2.5+dfsg1-6/debian/patches/link-against-wslay.patch/

Signed-off-by: Frank Denis <github@pureftpd.org>

rgacogne · 2018-11-22T09:10:48Z

pdns/dnsdistdist/doh.cc

+    uint16_t qtype, qclass;
+    unsigned int consumed = 0;
+    DNSName qname(query, len, sizeof(dnsheader), false, &qtype, &qclass, &consumed);
+    DNSQuestion dq(&qname, qtype, qclass, consumed, &du->dest, &du->remote, dh, 1500, len, false, &queryRealTime);


The 1500 here is wrong, it pretends that there is at least 1500 bytes available from the address pointed to by dh, which is not true. Rules and handleEDNSClientSubnet() will very likely trigger an heap-based out-of-bounds write.

rgacogne · 2018-11-22T09:12:49Z

pdns/dnsdistdist/doh.cc

+      uint32_t allowExpired = ss ? 0 : g_staleCacheEntriesTTL;
+      boost::optional<Netmask> subnet;
+      char cquery[1500];
+      memcpy(cquery, query, du->query.length());


I don't think you need to copy the query here, unless I missing something, since either it will be overwritten by the cached response on a hit, or we won't use cquery again before leaving this block.

I also did not check whether the query could be larger than 1500 bytes, which would trigger a stack-based out-of-bounds write.

rgacogne · 2018-11-22T09:19:16Z

pdns/dnsdistdist/doh.cc

+      vector<uint8_t> stripped;      
+      rewriteResponseWithoutEDNS(du->query, stripped);
+      du->query.assign((const char*)&stripped[0], stripped.size());
+    }


The case of an incoming query with EDNS but without ECS is not handled here.

…usly

… side

…nsdist processing

…ion closed

rgacogne · 2019-03-12T09:47:29Z

Unless I'm mistaken, we always send the request received over DoH to the backend over UDP, advertising an EDNS Payload Size of 4096 (unless there was an existing EDNS OPT RR). Doesn't that mean that we will never be able to handle responses larger than 4096 bytes over DoH?

MaxWichern · 2019-04-03T15:38:46Z

Starting dnsdist one way results in an error, while another way it works fine.
"nohup dnsdist --supervised &" works without any issues.
"systemctl start dnsdist" results in:
"dnsdist: what(): DOH thread failed to launch: DOH server failed to listen on 10.142.0.3:443: Permission denied
systemd: dnsdist.service: main process exited, code=killed, status=6/ABRT"

The error message itself should be more specific.
And the error should not end with an abort.

rgacogne · 2019-04-03T15:41:25Z

@MaxWichern It should have been fixed by 9bad969, is that not the case?

MaxWichern · 2019-04-03T15:47:39Z

@MaxWichern It should have been fixed by 9bad969, is that not the case?

That was the initial thought, but it didn't.

Habbie · 2019-04-05T15:17:08Z

When #7676 gets merged, we need to extend that code to reload h2o certs as well.

rgacogne · 2019-04-23T12:08:39Z

Superseded by #7726.

ahupowerdns changed the title ~~Dnsdist doh~~ Dnsdist doh [DO NOT MERGE] Aug 31, 2018

ahupowerdns added enhancement dnsdist labels Aug 31, 2018

publicarray referenced this pull request in DNSCrypt/dnscrypt-website Oct 31, 2018

dnsdist supports DoH, dnscrypt-wrapper doesn't support DoT

46a63ed

Signed-off-by: Frank Denis <github@pureftpd.org>

ahupowerdns force-pushed the dnsdist-doh branch 2 times, most recently from c631ea0 to f5c2664 Compare November 9, 2018 09:44

rgacogne reviewed Nov 22, 2018

View reviewed changes

ahupowerdns and others added 19 commits January 10, 2019 11:24

doh wip

0bcd4f0

WIP - gets question to main dnsdist now

b7cd9b3

a yolocommit - but it works with firefox nightly now!

de432c2

more wip

aa42be6

move to event loop, it works now

c1e5cc8

comment

9159f40

improve logging, free memory

0ec6d8f

align with firefox

3f0dc0c

make doh configurable - addDOHLocal(address, certfile, keyfile)

f73116e

hook up multiple URLs to DOH

33436f5

start cleanup for supporting multiple DoH connections at the same time

4052c9a

we are now in theory safe for multiple DoH servers running simultaneo…

8d04cdc

…usly

add a lot of comments, remove some dead code, quiet a lot of debugging

e351e76

rebase

550835b

implement translation of DNS timeouts or drops into 500 error on http…

00bcd56

… side

use full key chain, plus now emit a 404 on a wrong URL

5e6b505

implement more error cases, populate client HTTP IP address to main d…

cae2515

…nsdist processing

DoH: Add detection for libh2o-evloop

d6d480e

Ensure building dnsdist without DoH is possible

beeb16e

ahupowerdns and others added 10 commits January 10, 2019 11:25

this should fix closed sessions

9e87100

fix double cleanup when a request did not die from an error

8f4ed68

fix crashes when using the packet cache with DoH

cd16c8c

prevent crash on use of req in dnsdist processDOHQuery once http sess…

a525a0a

…ion closed

add TLS-version counters, plus fix regexrule for http/2

9a1ae14

make idle timeout configurable for DoH

990d2dc

don't crash on non-TLS, some whitespace

3bc840a

for GET requests, deal with missing padding

5a421d0

zero out accept_ctx before use

dde96b0

clean up some merge debris

8457dbe

ahupowerdns force-pushed the dnsdist-doh branch from 3cb8b42 to 8457dbe Compare January 10, 2019 10:39

ahupowerdns and others added 9 commits January 10, 2019 11:58

further merge problem

51611f0

re-add .gitmodules

f7b0aa0

ASAN found that we did not always zero the DOHUnit part of IDState

6761a9a

silence some debugging output

c1167a5

dnsdist: show DOH support in --version output

579400d

dnsdist: Add DoH to CentOS 7 packages

9207437

dnsdist: add DoH to Debian Stretch packages

b4cf8e3

Merge remote-tracking branch 'origin/master' into dnsdist-doh

c79ce41

Bind DoH ports before dropping privileges

9bad969

pieterlexis added 2 commits March 19, 2019 10:25

Merge remote-tracking branch 'origin/master' into dnsdist-doh

03c2a7b

Merge remote-tracking branch 'origin/master' into dnsdist-doh

6382609

rgacogne mentioned this pull request Apr 15, 2019

dnsdist: Add DNS over HTTPS support based on libh2o #7726

Merged

7 tasks

rgacogne closed this Apr 23, 2019

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dnsdist doh [DO NOT MERGE] #6911

Dnsdist doh [DO NOT MERGE] #6911

ahupowerdns commented Aug 31, 2018

johndickinson commented Sep 6, 2018 •

edited by Habbie

Loading

pieterlexis commented Sep 6, 2018

johndickinson commented Sep 6, 2018

zeha commented Sep 6, 2018

rgacogne Nov 22, 2018

rgacogne Nov 22, 2018

rgacogne Nov 22, 2018

rgacogne Nov 22, 2018

rgacogne commented Mar 12, 2019

MaxWichern commented Apr 3, 2019

rgacogne commented Apr 3, 2019

MaxWichern commented Apr 3, 2019

Habbie commented Apr 5, 2019

rgacogne commented Apr 23, 2019

Dnsdist doh [DO NOT MERGE] #6911

Dnsdist doh [DO NOT MERGE] #6911

Conversation

ahupowerdns commented Aug 31, 2018

Short description

Checklist

johndickinson commented Sep 6, 2018 • edited by Habbie Loading

pieterlexis commented Sep 6, 2018

johndickinson commented Sep 6, 2018

zeha commented Sep 6, 2018

rgacogne Nov 22, 2018

Choose a reason for hiding this comment

rgacogne Nov 22, 2018

Choose a reason for hiding this comment

rgacogne Nov 22, 2018

Choose a reason for hiding this comment

rgacogne Nov 22, 2018

Choose a reason for hiding this comment

rgacogne commented Mar 12, 2019

MaxWichern commented Apr 3, 2019

rgacogne commented Apr 3, 2019

MaxWichern commented Apr 3, 2019

Habbie commented Apr 5, 2019

rgacogne commented Apr 23, 2019

johndickinson commented Sep 6, 2018 •

edited by Habbie

Loading