Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Add option to open listener sockets in SO_REUSEPORT mode #244
These commits add a listener.reuseport option (which is disabled by default) that enables sniproxy to open the listener sockets in SO_REUSEPORT mode to allow multiple processes to bind to the same ip:port pair.
We gain kernel-based load-balancing of incoming connections that are evenly distributed between several sniproxy instances without the use of any external http/tcp proxy.
Interesting, I'm not able to look this over in depth now, but will in the next few days. How would this compare to forking off several child worker processes after the listening socket has been opened each running an event loop accepting new connections? My first thought is that it would be better to hide the multiprocess complexity than expose it to the user and hope they get it right.
I'm also curious what sort of scaling you were obtaining with a single process which instigated this improvement.
I look forward to hearing back, thank you!
Well, you can look on Nginx benchmarks that were performed when they've introduced this feature as a replacement for the same multiprocess paradigm that you propose: https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/ (tl;dr they've gained 20% reduced latency, 13x reduced latency jitter and 2x increased request rate)
We can use nginx model to fork workers opening their sockets in SO_REUSEPORT to hide the complexity of running several sniproxy manually, but i didn't have time to investigate the internals of sniproxy to implement this, so did a quick modification only.
The kernel balances incoming connections between processess consistently using hash over 4-tuple (src ip-port, dst ip-port) and it works fine even for UDP sockets.
We're using sniproxy+nginx to proxy https/http connections to certain domains in a large wireless network to allow these domains to work before authentication and keep our ACLs small and static.
Our main problem was not the scalability of sniproxy, but the limit of 65k TCP ports that is imposed on single IP-address that sniproxy uses as the source to initiate it's tunnels. So we needed multiple instances of sniproxy to listen on the same ip:port pair and each instance was using different source IP address to overcome this limitation. We could've placed HAProxy or something similar between clients and sniproxy in TCP mode, but SO_REUSEPORT is far more efficient and simple.
As for the CPU limitation, we've seen about 45% usage of a single core (Xeon 26xx v3 i think) by a single sniproxy process handling ~150Mbit TLS traffic, so when the traffic increased we could've easily hit 100% ceiling in a single thread/process model. So now we have two sniproxy processess in SO_REUSEPORT mode handling 300Mbit divided evenly between them. Although the main bottleneck is not traffic itself, but the rate of TLS handshakes that sniproxy has to decode and process, but i don't have connections/sec numbers at hand right now...
Thanks for getting back so quickly to my earlier questions. The Nginx article was quite interesting and suprirsing. Sorry it took so long to get back again.
I'm onboard with merging this feature, but still a little concerned with providing proper guidance on when to use it.
I ran into the same outbound ephemeral port range limitation with weighttp, and would be open to extending the configuration syntax allow specifying multiple backend source IPs to provide a more user friendly way of overcoming ephemeral port exhaustion.
There are still a few minor issues and the CI build is failing. I'm seeing a mix of tabs and spaces, and would like to introduce a new test (starting with functional_test) to test the two sniproxy daemon deployment, in addition if move parse boolean into the config module we can introduce a config_test case.
I'm happy to make these changes and merge this myself, but will not get to it until this evening (assuming I have internet access).