Add command line arguments to control incoming/outgoing send/receive socket OS buffer size

[notsure2]

Hello,
For advanced use case scenarios such as using shadowsocks as TPROXY in redir mode to bridge servers together, a bufferbloat effect happens that causes reduction in performance due to incorrect assumptions made by the OS when auto tuning buffers. To fix this problem manual control on the buffers is needed.

In addition, some users may like to have control on this as well for ss-local and ss-server for TCP manual tuning.

I implemented this for shadowsocks-libev. Is it possible to implement this for shadowsocks-rust ?
See shadowsocks/shadowsocks-libev#2781

@zonyitoo

[zonyitoo]

Well, its quite easy. But have you tried and verify that it is working on your env? Is that really matter for a proxy server?

[notsure2]

@zonyitoo yes I have tested this very thoroughly and I am using these options myself as daily driver 24x7 in server shadowsocks bridge. I have tested different values and measured change in TCP behavior with them (eg: performance of multisegment TCP vs single segment TCP download and upload).

Yes it matters for some use advanced cases especially when chaining proxies or chaining vpn to proxy or vice versa. It should be optional, most users don't need to use it for normal case of client PC connecting to a shadowsocks server as the default OS autotuning will handle it correctly.

[notsure2]

You can test that it's working by using Linux ss utility like this: ss -m -n state established dst destination-ip:port while making a download through the proxy chain and seeing the values of rb and tb (the buffer sizes).

[zonyitoo]

How about sndbuf / rcvbuf for udp sockets?

[zonyitoo]

Ah.. tokio didn't provide APIs for SO_SNDBUF and SO_RCVBUF. https://docs.rs/tokio/0.3.6/tokio/net/struct.TcpStream.html

So I have to implement it from scratch.

[notsure2]

For UDP there is no flow control or congestion avoidance or slow start or back off mechanism like TCP. In case of a TCP inside UDP flow which has the normal OS buffers (note: there is also no OS autotuning for UDP), it will feel any delays normally and I think it will adjust fine. I think the default OS buffer control for it are sufficient and no controls for it are necessary.

The problem is specifically for TCP because in case of TPROXY, the 2 sides of the proxy are highly asymmetrical (one side is very fast access to the chained server, and one side is the slow client).

Since from the OS Point of view, the connection between TPROXY shadowsocks and the chain server target is one connection, and the connection between the client and the main server is a completely different connection, the OS ramps up the buffers of one side too high than the other side, leading to a unique bufferbloat-similar phenomena where the packets are queued in the localhost shadowsocks redir buffer instead of being dropped due to congestion at the client side, and this causes confusion in the TCP congestion avoidance mechanism especially visible in case of multiple TCP connections from the same client to the same main server leading to poor overall network throughput.

@zonyitoo tokio doesn't let you reach socket fd and setsockopt on it directly ? or setsockopt before converting socket object to stream object.

[manjuprajna]

@notsure2
Thank you for your contribution, I am using ss-redir tproxy mode on arm/x86 openwrt client connecting to ss-libev server, so after your merge how can I tune my server/client side to optimize performance? do I need to add some lines on both config.json files of client/server side?

[zonyitoo]

Done. But I haven't tested yet. Still waiting for #354 .

[rwasef1830]

@manjuprajna If the connection between your PC and your router is faster than the connection between your router and the shadowsocks server, no tuning is needed. But if your connection to your own router is slow and the connection between the router and the remote server is much faster, then tuning is needed. You need to put in the config the same name as the command line arguments but replace hyphen with underscore. Test with a small buffer (512 KB) and increase by multiples, use any site that calculates TCP buffer size to get an estimate and test. Anyway, let's not hijack this bug report.

[zonyitoo]

Please test if the latest master works for you.

[notsure2]

ok I tested ss-local in redir mode talking to a chained ss-server with iperf, tcp and udp, forward and reverse, single and multiple segments and all seems to be ok. @zonyitoo

[zonyitoo]

Then problem solved. Will be released with v1.9.0.

[notsure2]

@zonyitoo you misunderstood me, I mean that the code works, not that this bug is solved. This issue is still there and the command line arguments are still needed. It is still needed to control the tcp buffers for send/receive incoming and outgoing to get optimal performance.

[zonyitoo]

Does the code that included in the current master solve your problem?

[notsure2]

Ok I didn't notice you already included the new config in master, sorry, I will retest with them now, are they supported in the config json ? or only in command line @zonyitoo

[zonyitoo]

Currently only in command line.

[notsure2]

there's a typo in the help text:

--outbound-redv-buffer-size <OUTBOUND_RECV_BUFFER_SIZE>    Set outbound sockets' SO_RCVBUF option
--outbound-send-buffer-size <OUTBOUND_SEND_BUFFER_SIZE>    Set outbound sockets' SO_SNDBUF option

redv instead of recv

[zonyitoo]

Oh. That is a bug. I tested it with C-V and never noticed that :(

[notsure2]

inbound parameter same typo also. and it accept the wrong name, if you write the correct name it rejects execution.
Ok i will recompile and retest.

[notsure2]

@zonyitoo Thank you so much sir. The buffer controls are working and I was able to achieve the expected level of performance using chained tproxy shadowsocks servers.