Health checks for targets that don't speak HTTP (WebSocket-only services) #231
Unanswered
spinosa
asked this question in
Ideas and Issue Triage
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
The problem
Health checks send an HTTP GET and require a 2xx. A WebSocket-only target has no HTTP endpoint to offer — it answers a plain GET by closing the connection — so it can never become healthy.
This comes up with accessories, which are frequently not web apps. An MQTT broker fronted as
wss://is the case I ran into.The usual workaround is to stand up a second, HTTP-only listener beside the real one purely so that something can be checked. That proves the second listener is up, and nothing about the port clients actually use — the two can diverge exactly when it matters.
Proposed solution
A
--health-check-protocol websocketoption that sends the WebSocket opening handshake and treats101 Switching Protocolsas healthy, so the check exercises the same listener as real traffic:Plus
--health-check-websocket-subprotocolto setSec-WebSocket-Protocol, which some servers require before they will complete the handshake.Default stays
http, and an unset value behaves ashttp, so nothing changes for existing users. An unrecognised protocol is rejected rather than silently falling back to HTTP.Implementation
I have this working and tested, if it's of interest:
main...spinosa:kamal-proxy:websocket-health-checks
Upgrade, theConnectiontoken,Sec-WebSocket-Acceptderived from a fresh random nonce, and no subprotocol that wasn't offeredRouter.DeployService, matching theServiceOptions.Validateconventiongo vetclean, race-clean; verified against a real Mosquitto brokerOne subtlety I'd want a second opinion on: on a protocol switch
Response.Bodyis the upgraded connection, so the usual drain has to be skipped for 101 — draining it blocks until the peer sends something. There's a regression test that holds the connection open, since anhttptestserver closes on handler return and hides the problem entirely.There's a companion change on the Kamal side to pass the flags through from
deploy.yml, which would need a release containing these flags before it can land.Following CONTRIBUTING.md by opening this as a discussion first rather than a PR — happy to open one if you'd like to take it.
Written by me in collaboration with Claude. I've reviewed all of it myself — quickly, but all of it.
All reactions