Description
I'm trying to setup Caddy as a reverse proxy for DoH but there's a problem on the backend connection between Caddy and the Bind9 server, which involves HTTP/2
If I use dig for instance on a client and point it directly to the Bind server, I get a DNS response
But when I try to reverse proxy this, the dig command fails
This is the config on Caddy
doh.homelab.lan {
tls internal
reverse_proxy {
to bind9-internal.local:443
transport http {
tls
tls_insecure_skip_verify
versions h2
}
header_up Host {http.reverse_proxy.upstream.hostport}
header_up X-Forwarded-For {http.request.remote.host}
header_up X-Forwarded-Proto {http.request.scheme}
}
}
I did a check with an openssl s_client command from Caddy to Bind and I could see "ALPN protocol: h2" in the output and I've also set that in the config to make sure it gets used
When I checked the Caddy logs I found entries like this
{"level":"debug","ts":1751894451.7522008,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"bind9-internal.local:443","duration":0.005224311,"request":{"remote_ip":"192.168.200.10","remote_port":"43083","client_ip":"192.168.200.10","proto":"HTTP/2.0","method":"POST","host":"bind9-internal.local:443","uri":"/dns-query","headers":{"Cache-Control":["no-cache, no-store, must-revalidate"],"Accept":["application/dns-message"],"Content-Length":["57"],"User-Agent":[""],"X-Forwarded-Host":["doh.homelab.lan:443"],"Content-Type":["application/dns-message"],"X-Forwarded-For":["192.168.200.10"],"X-Forwarded-Proto":["https"],"Via":["2.0 Caddy"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","server_name":""}},"error":"net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00d""}
{"level":"error","ts":1751894451.7522838,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00d"","request":{"remote_ip":"192.168.200.10","remote_port":"43083","client_ip":"192.168.200.10","proto":"HTTP/2.0","method":"POST","host":"doh.homelab.lan:443","uri":"/dns-query","headers":{"Accept":["application/dns-message"],"Content-Length":["57"],"Cache-Control":["no-cache, no-store, must-revalidate"],"Content-Type":["application/dns-message"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","server_name":""}},"duration":0.005462997,"status":502,"err_id":"2tfifr2c1","err_trace":"reverseproxy.statusError (reverseproxy.go:1390)"}
With some help debugging the stream sent by Bind, the problem I'm told could be that Caddy is having an issue with SETTINGS_MAX_CONCURRENT_STREAMS set to 0
The software versions are Caddy 2.10.0 and Bind9 9.18.33-1~deb12u2-Debian (Extended Support Version)