FS#3231 - firewall3: mtu_fix should apply mss clamping for both outgoing and incoming syn packets

[openwrt-bot]

giner:

TL;DR When mtu_fix is enabled mss side is only altered for incoming traffic but not outgoing. This affects clients with pmtu disabled.

The issue is quite difficult to reproduce as it requires specific conditions however it happens in real setups quite regularly.

The conditions are bellow:

• OpenWrt router working in "router" mode with IPv4 traffic NATed towards Internet
• Upstream is connected by means of something that has reduced MTU (PPPoE, DS-LITE, etc)
• mtu_fix is applied to WAN interfaces

The issue:

• MSS from a client towards a server in the internet is not reduced to fit reduced MTU if the client has PMTU disabled (one example of such a client - LineageOS 16.x). As a result the packets of maximums MSS towards the server will be dropped.

Solution:

• Enable mss clamping form incoming SYN packets on the interface with mtu_fix enabled

iptables without the fix:

iptables-save -t mangle | grep clamp

-A FORWARD -o ds-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu


iptables with the fix:

iptables-save -t mangle | grep clamp
-A FORWARD -o ds-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -i ds-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu


TCP session initiation without the fix (WAN mtu is 1452):

tcpdump -ni wlan0 "(tcp[tcpflags] & (tcp-syn) != 0) and (host www.twitter.com)"

11:05:25.531925 IP 192.168.1.141.35040 > 104.244.42.129.80: Flags [S], seq 2878102236, win 64240, options [mss 1460,sackOK,TS val 2138517727 ecr 0,nop,wscale 7], length 0
11:05:25.536034 IP 104.244.42.129.80 > 192.168.1.141.35040: Flags [S.], seq 180174328, ack 2878102237, win 28960, options [mss 1420,sackOK,TS val 2685758451 ecr 2138517727,nop,wscale 8], length 0


Note ** mss 1420 ** in the syn packet towards the client. This make the client consider 1420 being the maximum possible mss however of the client tries to send such a packet with DF flag set it will be silently dropped.

TCP session initiation the fix (WAN mtu is 1452):

tcpdump -ni wlan0 "(tcp[tcpflags] & (tcp-syn) != 0) and (host www.twitter.com)"

11:02:42.062918 IP 192.168.1.141.34954 > 104.244.42.129.80: Flags [S], seq 4079972940, win 64240, options [mss 1460,sackOK,TS val 2138354260 ecr 0,nop,wscale 7], length 0
11:02:42.067116 IP 104.244.42.129.80 > 192.168.1.141.34954: Flags [S.], seq 416388380, ack 4079972941, win 28960, options [mss 1412,sackOK,TS val 920483133 ecr 2138354260,nop,wscale 8], length 0


Note ** mss 1412 ** in the syn packet towards the client. This is a real maximum for mss.

[openwrt-bot]

bjonglez:

Good bug hunting.

For the record, it's not that obvious that the fix works: it depends on the behaviour of ''--clamp-mss-to-pmtu''. If it only looks at the PMTU towards the destination IP of the packets, it won't work for incoming packets.

It turns out that ''--clamp-mss-to-pmtu'' looks at both source and destination IP and that's why it works:

--clamp-mss-to-pmtu Automatically clamp MSS value to (path_MTU - 40 for IPv4; -60 for IPv6). This may not function as desired where asymmetric routes with differing path MTU exist — the kernel uses the path MTU which it would use to send packets from itself to the source and destination IP addresses. Prior to Linux 2.6.25, only the path MTU to the destination IP address was considered by this option; subsequent kernels also consider the path MTU to the source IP address.

[openwrt-bot]

yousong:

Last time I saw this issue was when reviewing a wireguard setup. IP fragmentation
in the client->server direction was very frequent in the pppoe-wan interface. I think this is why the old way mostly works, fragmentation in one direction.

At that time I fixed it by setting a correct mtu value (16 multiples) on the local openwrt side, and having mss clamp in both directions on the remote side.