Firewall Shaper causes IPv6 address loss on WAN

[meyergru]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

If one uses the traffic shaper like described here: https://maltechx.de/en/2021/03/opnsense-setup-traffic-shaping-and-reduce-bufferbloat/ and elsewhere, the IPv6 address of the WAN interfaces gets lost when an outbound congestion occurs. IPv4 works fine.

This is especially problematic with dynamic IPv6 prefixes and CG-NAT / DS-Lite. The german ISP Deutsche Glasfaser is a prominent example of this.

To Reproduce

There are multiple reports of this behaviour on the OpnSense forums:

https://forum.opnsense.org/index.php?topic=27247.0
https://forum.opnsense.org/index.php?topic=39624.0
https://forum.opnsense.org/index.php?topic=32912

Expected behavior

The IPv6 connectivity should not be disturbed by traffic shaping.

Describe alternatives you considered

In two of the threads, a potential workaround is to use specific IPv6 ranges for the source of the rules, which is impossible with dynamic IPv6 prefixes, though.

Relevant log files

See links...

Environment

OPNsense 22.x to 24.1.4

[Wireheadbe]

I have shaping working on both my WAN links - and both have IPv6 enabled. The only difference between my rules and the rules in the guide, is that I do not have a mask set on the queue, and only use direction on the rule.

Could you test what that does?

[prolix-oc]

Since @Wireheadbe's question was posed and did not receive an answer, I will follow up since this issue seemingly gets no activity (for a rather large issue if I can say so):

Just so you can rule out any hardware issues...

Hardware setup:

• Intel Core i7-2600
• 8GB DDR3-1333
• 250GB Micron SSD formatted for ZFS
• Mellanox MCX4121A-ACAT for LAN
• Intel I226-V for WAN
• IPv4 and IPv6 DHCP working perfectly, until...

You simply turn on any ip protocol based shaping rules. Leaving it to ipv4 works and obtains a DHCPv6 address from the ISP. Leaving it to tcp and udp in paired rulings for both directions works [UPDATE: it does not. ICMP is technically TCP traffic too, and I forgot that fact when I attempted to re-obtain a lease]. Any time ip is left on, it includes the shaping of ICMPv6 and ICMP packets, which fragment the response from the gateway.

My running theory is that the packets are getting fragmented by the shaper and not reassembled correctly. This much I can confirm by seeing the solicit packets missing bytelengths in the system logs, then restarting the solicit process to get a valid packet.

The most obvious answer to this would simply exclude ALL ICMP traffic from the shaper when using the ip protocol. We all know not to disable ICMP at this point, right? If this is too much of an overreach for the devs (for some reason I can't begin to imagine), just adding an advanced option to "exclude ICMPv4 and v6 traffic" option would solve this too.

Let me know if I just said a lot of nothing, but this isn't my first rodeo fighting ICMP solicit issues with traffic shaping on opnSense. For some reason on my previous pfSense install, that was never an issue.