Skip to content

reverse proxy: http transport no error emitted when invalid http versions are provided #7111

Closed
@dsmckone1528

Description

@dsmckone1528

I'm trying to setup Caddy as a reverse proxy for DoH but there's a problem on the backend connection between Caddy and the Bind9 server, which involves HTTP/2

If I use dig for instance on a client and point it directly to the Bind server, I get a DNS response

But when I try to reverse proxy this, the dig command fails

This is the config on Caddy

doh.homelab.lan {
	tls internal
	reverse_proxy {
		to bind9-internal.local:443
		transport http {
			tls
			tls_insecure_skip_verify
			versions h2
		}
		header_up Host {http.reverse_proxy.upstream.hostport}
		header_up X-Forwarded-For {http.request.remote.host}
		header_up X-Forwarded-Proto {http.request.scheme}
	}
}

I did a check with an openssl s_client command from Caddy to Bind and I could see "ALPN protocol: h2" in the output and I've also set that in the config to make sure it gets used

When I checked the Caddy logs I found entries like this

{"level":"debug","ts":1751894451.7522008,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"bind9-internal.local:443","duration":0.005224311,"request":{"remote_ip":"192.168.200.10","remote_port":"43083","client_ip":"192.168.200.10","proto":"HTTP/2.0","method":"POST","host":"bind9-internal.local:443","uri":"/dns-query","headers":{"Cache-Control":["no-cache, no-store, must-revalidate"],"Accept":["application/dns-message"],"Content-Length":["57"],"User-Agent":[""],"X-Forwarded-Host":["doh.homelab.lan:443"],"Content-Type":["application/dns-message"],"X-Forwarded-For":["192.168.200.10"],"X-Forwarded-Proto":["https"],"Via":["2.0 Caddy"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","server_name":""}},"error":"net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00d""}
{"level":"error","ts":1751894451.7522838,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00d"","request":{"remote_ip":"192.168.200.10","remote_port":"43083","client_ip":"192.168.200.10","proto":"HTTP/2.0","method":"POST","host":"doh.homelab.lan:443","uri":"/dns-query","headers":{"Accept":["application/dns-message"],"Content-Length":["57"],"Cache-Control":["no-cache, no-store, must-revalidate"],"Content-Type":["application/dns-message"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","server_name":""}},"duration":0.005462997,"status":502,"err_id":"2tfifr2c1","err_trace":"reverseproxy.statusError (reverseproxy.go:1390)"}

With some help debugging the stream sent by Bind, the problem I'm told could be that Caddy is having an issue with SETTINGS_MAX_CONCURRENT_STREAMS set to 0

The software versions are Caddy 2.10.0 and Bind9 9.18.33-1~deb12u2-Debian (Extended Support Version)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions