client: only set TCP socket buffer size on macOS

[xv-thomas-leong]

Setting SO_SNDBUF/SO_RCVBUF on TCP disables kernel buffer autotuning, capping the bandwidth-delay product and throttling high-RTT links. UDP still needs explicit sizing since it has no autotuning.

However, macOS benefits from explicit buffers based on real-world testing.

How Has This Been Tested?

CI tested and manually tested.

Performance difference on Lightway TCP when socket buffer is set/not set over a high RTT connection:

Linux

Set Socket Buffer?	Idle Latency (ms)	Download (Mbps)	Download Latency (ms)	Upload (Mbps)	Upload Latency (ms)	Packet Loss (%)
Yes	190.91	11.22	662.88	8.59	360.53	0.0
No	190.50	160.86 👑	845.22	59.29 👑	296.45	0.0

macOS

Set Socket Buffer?	Idle Latency (ms)	Download (Mbps)	Download Latency (ms)	Upload (Mbps)	Upload Latency (ms)	Packet Loss (%)
Yes	190.20	166.05	792.47	249.29 👑	285.27	0.97
No	188.85	167.37	730.95	101.17	278.61	0.0

Windows

Set Socket Buffer?	Idle Latency (ms)	Download (Mbps)	Download Latency (ms)	Upload (Mbps)	Upload Latency (ms)	Packet Loss (%)
Yes	192.89	326.50	680.44	256.90	316.64	0.0
No	190.36	457.56 👑	514.88	450.68 👑	379.42	0.0

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• The correct base branch is being used, if not main

[github-actions]

Code coverage summary for 9a2191d:

Filename                                                          Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
lightway-app-utils/src/args/cipher.rs                                   5                 5     0.00%           1                 1     0.00%           5                 5     0.00%           0                 0         -
lightway-app-utils/src/args/connection_type.rs                          8                 8     0.00%           2                 2     0.00%           8                 8     0.00%           0                 0         -
lightway-app-utils/src/args/duration.rs                               121                22    81.82%          11                 5    54.55%          64                15    76.56%           0                 0         -
lightway-app-utils/src/args/ip_map.rs                                  27                27     0.00%           5                 5     0.00%          19                19     0.00%           0                 0         -
lightway-app-utils/src/args/keyshare.rs                                 5                 5     0.00%           1                 1     0.00%           5                 5     0.00%           0                 0         -
lightway-app-utils/src/args/logging.rs                                 37                37     0.00%           3                 3     0.00%          31                31     0.00%           0                 0         -
lightway-app-utils/src/args/nonzero_duration.rs                       128                24    81.25%          10                 4    60.00%          68                16    76.47%           0                 0         -
lightway-app-utils/src/connection_ticker.rs                           229                17    92.58%          28                 4    85.71%         126                15    88.10%           0                 0         -
lightway-app-utils/src/dplpmtud_timer.rs                              209                13    93.78%          22                 4    81.82%         117                11    90.60%           0                 0         -
lightway-app-utils/src/event_stream.rs                                 19                 0   100.00%           3                 0   100.00%          11                 0   100.00%           0                 0         -
lightway-app-utils/src/packet_codec.rs                                  2                 2     0.00%           1                 1     0.00%           1                 1     0.00%           0                 0         -
lightway-app-utils/src/sockopt/ip_mtu_discover.rs                      81                81     0.00%           6                 6     0.00%          72                72     0.00%           0                 0         -
lightway-app-utils/src/sockopt/ip_pktinfo.rs                           14                14     0.00%           1                 1     0.00%          16                16     0.00%           0                 0         -
lightway-app-utils/src/tun.rs                                         284               284     0.00%          31                31     0.00%         178               178     0.00%           0                 0         -
lightway-app-utils/src/utils.rs                                        21                21     0.00%           1                 1     0.00%          11                11     0.00%           0                 0         -
lightway-client/src/config.rs                                         292                96    67.12%          13                 6    53.85%         198                59    70.20%           0                 0         -
lightway-client/src/dns_manager.rs                                     18                18     0.00%           4                 4     0.00%          16                16     0.00%           0                 0         -
lightway-client/src/io/inside/tun.rs                                   82                82     0.00%          12                12     0.00%          60                60     0.00%           0                 0         -
lightway-client/src/io/outside.rs                                      14                14     0.00%           2                 2     0.00%          12                12     0.00%           0                 0         -
lightway-client/src/io/outside/tcp.rs                                  89                89     0.00%          12                12     0.00%          56                56     0.00%           0                 0         -
lightway-client/src/io/outside/udp.rs                                 167               167     0.00%          17                17     0.00%         108               108     0.00%           0                 0         -
lightway-client/src/io/outside/udp_batch_receiver.rs                   61                11    81.97%           6                 1    83.33%          56                12    78.57%           0                 0         -
lightway-client/src/keepalive.rs                                      615                44    92.85%          55                 6    89.09%         330                25    92.42%           0                 0         -
lightway-client/src/lib.rs                                           1246               670    46.23%         102                53    48.04%         835               489    41.44%           0                 0         -
lightway-client/src/main.rs                                           351               351     0.00%          20                20     0.00%         233               233     0.00%           0                 0         -
lightway-client/src/platform/linux/dns_manager.rs                     118                91    22.88%          14                 9    35.71%          87                62    28.74%           0                 0         -
lightway-client/src/platform/linux/dns_manager/direct_file.rs         106                42    60.38%          12                 6    50.00%          65                29    55.38%           0                 0         -
lightway-client/src/platform/linux/dns_manager/resolvconf.rs           61                61     0.00%           8                 8     0.00%          52                52     0.00%           0                 0         -
lightway-client/src/platform/linux/dns_manager/resolvectl.rs           41                41     0.00%           5                 5     0.00%          34                34     0.00%           0                 0         -
lightway-client/src/route_manager.rs                                 1190               212    82.18%          77                 8    89.61%         663               117    82.35%           0                 0         -
lightway-core/src/borrowed_bytesmut.rs                                373                 0   100.00%          24                 0   100.00%         185                 0   100.00%           0                 0         -
lightway-core/src/builder_predicates.rs                                24                12    50.00%           4                 2    50.00%          24                12    50.00%           0                 0         -
lightway-core/src/cipher.rs                                            13                 0   100.00%           2                 0   100.00%          10                 0   100.00%           0                 0         -
lightway-core/src/connection.rs                                      1663               826    50.33%          72                27    62.50%        1173               552    52.94%           0                 0         -
lightway-core/src/connection/builders.rs                              252                46    81.75%          22                 8    63.64%         259                53    79.54%           0                 0         -
lightway-core/src/connection/dplpmtud.rs                             1741                81    95.35%          63                 0   100.00%         830                 7    99.16%           0                 0         -
lightway-core/src/connection/expresslane.rs                            27                11    59.26%           4                 1    75.00%          36                 7    80.56%           0                 0         -
lightway-core/src/connection/fragment_map.rs                          366                 6    98.36%          25                 0   100.00%         254                 3    98.82%           0                 0         -
lightway-core/src/connection/io_adapter.rs                            526                23    95.63%          34                 5    85.29%         276                21    92.39%           0                 0         -
lightway-core/src/connection/key_update.rs                             34                13    61.76%           5                 0   100.00%          38                19    50.00%           0                 0         -
lightway-core/src/context.rs                                          214                51    76.17%          28                11    60.71%         235                64    72.77%           0                 0         -
lightway-core/src/context/ip_pool.rs                                    8                 3    62.50%           1                 0   100.00%           5                 0   100.00%           0                 0         -
lightway-core/src/context/server_auth.rs                               37                37     0.00%           4                 4     0.00%          26                26     0.00%           0                 0         -
lightway-core/src/encoding_request_states.rs                            3                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
lightway-core/src/io.rs                                                10                10     0.00%           3                 3     0.00%           9                 9     0.00%           0                 0         -
lightway-core/src/keyshare.rs                                           5                 0   100.00%           1                 0   100.00%           5                 0   100.00%           0                 0         -
lightway-core/src/lib.rs                                                9                 0   100.00%           3                 0   100.00%           9                 0   100.00%           0                 0         -
lightway-core/src/metrics.rs                                           51                41    19.61%          21                17    19.05%          47                37    21.28%           0                 0         -
lightway-core/src/packet.rs                                            38                10    73.68%           4                 1    75.00%          30                 6    80.00%           0                 0         -
lightway-core/src/plugin.rs                                           303                13    95.71%          21                 3    85.71%         145                 7    95.17%           0                 0         -
lightway-core/src/tls/mod.rs                                            4                 4     0.00%           1                 1     0.00%           3                 3     0.00%           0                 0         -
lightway-core/src/utils.rs                                            334                26    92.22%          22                 2    90.91%         165                17    89.70%           0                 0         -
lightway-core/src/version.rs                                           93                 0   100.00%          17                 0   100.00%          82                 0   100.00%           0                 0         -
lightway-core/src/wire.rs                                             438                42    90.41%          29                 1    96.55%         235                12    94.89%           0                 0         -
lightway-core/src/wire/auth_failure.rs                                 27                 1    96.30%           3                 0   100.00%          17                 0   100.00%           0                 0         -
lightway-core/src/wire/auth_request.rs                                684                57    91.67%          34                 1    97.06%         351                24    93.16%           0                 0         -
lightway-core/src/wire/auth_success_with_config_ipv4.rs               222                 3    98.65%          11                 0   100.00%         124                 0   100.00%           0                 0         -
lightway-core/src/wire/data.rs                                         51                 1    98.04%           5                 0   100.00%          33                 0   100.00%           0                 0         -
lightway-core/src/wire/data_frag.rs                                   127                 1    99.21%          14                 0   100.00%          80                 0   100.00%           0                 0         -
lightway-core/src/wire/encoding_request.rs                             82                 2    97.56%           6                 0   100.00%          42                 1    97.62%           0                 0         -
lightway-core/src/wire/encoding_response.rs                            82                 2    97.56%           6                 0   100.00%          42                 1    97.62%           0                 0         -
lightway-core/src/wire/expresslane_config.rs                          192                 1    99.48%          10                 0   100.00%         110                 0   100.00%           0                 0         -
lightway-core/src/wire/expresslane_data.rs                           1330                69    94.81%          48                 5    89.58%         679                37    94.55%           0                 0         -
lightway-core/src/wire/ping.rs                                         95                 2    97.89%           7                 0   100.00%          59                 0   100.00%           0                 0         -
lightway-core/src/wire/pong.rs                                        109                 2    98.17%           8                 0   100.00%          72                 0   100.00%           0                 0         -
lightway-core/src/wire/server_config.rs                                69                 2    97.10%           4                 0   100.00%          37                 0   100.00%           0                 0         -
lightway-server/src/auth.rs                                           273                49    82.05%          22                 6    72.73%         171                28    83.63%           0                 0         -
lightway-server/src/config.rs                                          26                26     0.00%           1                 1     0.00%          44                44     0.00%           0                 0         -
lightway-server/src/connection.rs                                     137               137     0.00%          11                11     0.00%         110               110     0.00%           0                 0         -
lightway-server/src/connection_manager.rs                             339               339     0.00%          40                40     0.00%         294               294     0.00%           0                 0         -
lightway-server/src/connection_manager/connection_map.rs              407                16    96.07%          21                 1    95.24%         228                 7    96.93%           0                 0         -
lightway-server/src/io/inside/tun.rs                                   44                44     0.00%           9                 9     0.00%          30                30     0.00%           0                 0         -
lightway-server/src/io/outside/tcp.rs                                  90                90     0.00%           9                 9     0.00%          65                65     0.00%           0                 0         -
lightway-server/src/io/outside/udp.rs                                 320               320     0.00%          16                16     0.00%         206               206     0.00%           0                 0         -
lightway-server/src/io/outside/udp/cmsg.rs                            197                53    73.10%          14                 5    64.29%         158                43    72.78%           0                 0         -
lightway-server/src/ip_manager.rs                                     542                50    90.77%          22                 4    81.82%         242                23    90.50%           0                 0         -
lightway-server/src/ip_manager/ip_pool.rs                             545                 0   100.00%          28                 0   100.00%         257                 0   100.00%           0                 0         -
lightway-server/src/lib.rs                                            298               298     0.00%          15                15     0.00%         198               198     0.00%           0                 0         -
lightway-server/src/main.rs                                           246               246     0.00%          10                10     0.00%         129               129     0.00%           0                 0         -
lightway-server/src/metrics.rs                                        254               250     1.57%          85                83     2.35%         215               211     1.86%           0                 0         -
lightway-server/src/statistics.rs                                     132                59    55.30%           8                 4    50.00%          90                39    56.67%           0                 0         -
uniffi-bindgen/src/uniffi-bindgen.rs                                    3                 3     0.00%           1                 1     0.00%           3                 3     0.00%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                               19130              5927    69.02%        1364               545    60.04%       11707              4115    64.85%           0                 0         -

✅ Region coverage 69% passes
✅ Line coverage 64% passes

[kp-mariappan-ramasamy]

We could also update the config documentation to reflect this change:

lightway/lightway-client/src/config.rs

Lines 172 to 184 in 5c6ded2

	#[patch(attribute(clap(long)))]
	#[patch(attribute(doc = "Socket send buffer size"))]
	#[schemars(extend("x-cfg" = "desktop"))]
	#[schemars(schema_with = "byte_size_schema")]
	/// ex: 1.5 MiB
	pub sndbuf: ByteSize,
	#[patch(attribute(clap(long)))]
	#[patch(attribute(doc = "Socket receive buffer size"))]
	#[schemars(extend("x-cfg" = "desktop"))]
	#[schemars(schema_with = "byte_size_schema")]
	/// ex: 1.5 MiB
	pub rcvbuf: ByteSize,

Or add a warn skipping these values in TCP

[xv-thomas-leong]

Good suggestion 🙏 added docs in config.rs