Anycast multi-region Caddy setup using QUIC's preferred address mechanism

[marten-seemann]

The following setup should (in theory) be possible: Multiple Caddy instances authoritative for the same (set of) domain(s) running on geographically distributed servers reachable under the same anycast IP address. The browser connects to the ~closest server, completes the QUIC handshake, and then migrates to the unicast address (to guarantee stable routing over the lifetime of the connection) using QUIC's Server Preferred Address mechanism.

The blog post on quic.video (search for anycast) has a good description and some diagrams.

Protocol-wise, all the required elements are in place. Implementation-wise, this would need proper support for preferred address in quic-go, and I'm still trying to confirm if major browsers have actually implemented this feature.

I'm opening this issue to document the idea, and to see if there's interest in making it happen. I could see setups where this would be very interesting, but I don't know how peoples' deployments look like, so maybe this is just a crazy idea.

[francislavoie]

Would this be something users need to opt-in for, or would it just kinda work automatically? What would the config surface look like, etc?

[marten-seemann]

All that the QUIC layer needs to know is the unicast IP address(es). These would have to be passed into quic-go somehow (API still TBD). I'm not sure if Caddy already has access to that somewhere.

Other than that, you'd have to configure your server such that they're all reachable at the same Anycast IP address, of course. This requires a bit more work than just spinning a single machine, but you'd get a geographically distributed server setup in return (which is one of the selling points of CDN operators).

[mholt]

So this would make it almost effortless to stand up your own CDN? Just deploy replicas of your site to your Caddy instances around the world and configure them with the right IP addresses and boom?

That would be insanely cool if so.

Anyway, definitely open to this.