bogons: blocking order for IPv6 et al

[xtof01]

Due to the move to PHP 7 in 17.1 Beta, there is a problem when trying to configure IPv6 interface tracking.
On the interface configuration page, after setting "IPv6 Configuration Type" to "Track Interface" and selecting the IPv6 interface to track, you can enter an IPv6 prefix ID in order to select a specific prefix from the (potentially larger) prefix delegated to you.
However, entering any number different from 0 always results in getting the error message

You must enter a valid hexadecimal number for the IPv6 prefix ID.

The reason is that in line 694 of interfaces.php the function is_numeric is used to check the prefix ID, with a "0x" prefix prepended to the form input.
Unfortunately, as stated in the PHP documentation, support for hex notation has been removed from is_numeric in PHP 7. Therefore, is_numeric is always false.

[fichtner]

@xtof01 Oh thanks very much for this! Can you try the following patch?

# opnsense-patch 3bbe389

[xtof01]

Thanks, that fixed it.

There is still a problem with Prefix Delegation, but I'm still trying to figure that out ...

[fichtner]

@xtof01

Can you revert the following patches (c5fe0e4 8067e9b d3a6960) by running this on the shell?

[kgleason]

I'm going to try all of this tonight and see if it resolves my IPv6 issues. I've been experiencing something similar.

[xtof01]

@kgleason I wouldn't recommend it 😃

@fichtner
That didn't go as planned. The first patch (c5fe0e4) didn't get reverted, but applied normally. Unfortunately I didn't notice that right away, but only when it was already too late.
patch-result.txt
I manually fixed the failed hunk from the last patch and rebooted, which produced the following result:

Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring QinQ interfaces...done.
Configuring GUEST interface...
Fatal error: Uncaught Error: Call to undefined function legacy_netgraph_detach() in /usr/local/etc/inc/interfaces.inc:2793
Stack trace:
#0 /usr/local/etc/inc/interfaces.inc(1051): interface_configure('opt2', false, false, true)
#1 /usr/local/etc/rc.bootup(115): interfaces_configure(true)
#2 {main}
  thrown in /usr/local/etc/inc/interfaces.inc on line 2793

FreeBSD/amd64 (fw.localdomain) (ttyu0)

login: 

So I reverted the first patch (manually -- didn't have a working network connection) and tried again.
This time:

Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring QinQ interfaces...done.
Configuring GUEST interface...done.
Configuring LAN interface...done.
Configuring MGMT interface...done.
Configuring WAN interface...done.
Configuring WLAN interface...done.
Configuring INTBR interface...done.
Generating /etc/resolv.conf...done.
Configuring firewall.
Fatal error: Uncaught Error: Call to undefined function legacy_interfaces_details() in /usr/local/etc/inc/filter.inc:877
Stack trace:
#0 /usr/local/etc/inc/filter.inc(409): filter_generate_network_aliases()
#1 /usr/local/etc/rc.bootup(119): filter_configure_sync(true)
#2 {main}
  thrown in /usr/local/etc/inc/filter.inc on line 877

FreeBSD/amd64 (fw.localdomain) (ttyu0)

login: 

Since I didn't see a possibility to fix this (reverting the third patch removed legacy_interfaces_details), I stopped at this point and re-installed 17.1.b in order to get back to a working system.

One interesting thing though: Despite the system not booting completely and not routing any traffic, on the second boot it did successfully manage to request prefix delegation (I observed this with Wireshark), and it did assign a prefix to the internal tracking interface:

pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
        inet6 fe80::20d:b9ff:fe42:6170%pppoe0 prefixlen 64 scopeid 0xa 
        inet6 fe80::20d:b9ff:fe42:6172%pppoe0 prefixlen 64 scopeid 0xa 
        inet6 2003:73:e7f:9753:20d:b9ff:fe42:6170 prefixlen 64 autoconf 
        inet 91.17.210.185 --> 217.0.116.186  netmask 0xffffffff 
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:f5:d6:e7:3c:00
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 
        inet6 2003:73:e17:5301:f5:d6ff:fee7:3c00 prefixlen 64 
        inet6 fe80::1:1%bridge0 prefixlen 64 scopeid 0xb 
        nd6 options=1<PERFORMNUD>
        groups: bridge 
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: ath0_wlan1 flags=363<LEARNING,DISCOVER,EDGE,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 33333
        member: igb1 flags=363<LEARNING,DISCOVER,EDGE,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55

[fichtner]

Sorry, I forgot to mention how to reset the original package:

 # pkg install -f opnsense-devel

I will prepare a correct backport tomorrow, but @AdSchellevis did already indicate that it may indeed be another subtle PHP 7 issue and not related to direct changes.

[kgleason]

Well I didn't refresh this before I tried, but it didn't seem to affect anything. I did have some failures ... looks like perhaps Hunk #2 of 3 failed to apply. Still no IPv6.

I should probably clarify that my issue I think is less specific than this issue. I am able to enter a prefix (or rather choose one from the dropdown), but I'm just not getting any IPv6 address on my WAN.

[fichtner]

Ok, here we go:

# cd /root
# fetch https://pkg.opnsense.org/patches/pre-interfaces.lib.inc
# fetch https://pkg.opnsense.org/patches/post-interfaces.lib.inc

The 16.7.x state is this one:

# cp pre-interfaces.lib.inc /usr/local/etc/inc/interfaces.lib.inc

Some fixes for 17.1 went into this one:

# cp post-interfaces.lib.inc /usr/local/etc/inc/interfaces.lib.inc

Resetting the code is still done by:

# pkg install -f opnsense-devel

Thanks in advance,
Franco

[xtof01]

Okay, I hope I understood your instructions correctly.
Starting with a blank 17.1 Beta, I restored my configuration XML file. Then I applied patch 3bbe389, then configured interface tracking for my internal interface.
I did the two fetch commands.

Test 1
I copied post-interfaces.lib.inc to /usr/local/etc/inc/interfaces.lib.inc and rebooted.
Result:

Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring QinQ interfaces...done.
Configuring GUEST interface...
Fatal error: Uncaught Error: Call to undefined function legacy_netgraph_detach() in /usr/local/etc/inc/i3
Stack trace:
#0 /usr/local/etc/inc/interfaces.inc(1051): interface_configure('opt2', false, false, true)
#1 /usr/local/etc/rc.bootup(115): interfaces_configure(true)
#2 {main}
  thrown in /usr/local/etc/inc/interfaces.inc on line 2793

FreeBSD/amd64 (fw.localdomain) (ttyu0)

login: 

No network connectivity, interfaces didn't get configured.

Test 2
I copied pre-interfaces.lib.inc to /usr/local/etc/inc/interfaces.lib.inc and rebooted.
Result:

Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring QinQ interfaces...done.
Configuring GUEST interface...done.
Configuring LAN interface...done.
Configuring MGMT interface...done.
Configuring WAN interface...done.
Configuring WLAN interface...done.
Configuring INTBR interface...done.
Generating /etc/resolv.conf...done.
Configuring firewall.
Fatal error: Uncaught Error: Call to undefined function legacy_interfaces_details() in /usr/local/etc/in7
Stack trace:
#0 /usr/local/etc/inc/filter.inc(409): filter_generate_network_aliases()
#1 /usr/local/etc/rc.bootup(119): filter_configure_sync(true)
#2 {main}
  thrown in /usr/local/etc/inc/filter.inc on line 877

FreeBSD/amd64 (fw.localdomain) (ttyu0)

login: 

This is basically the same situation as in my post from two days ago: The interfaces are configured, the PPPoE link is established, even Prefix Delegation happens and the internal interface has an IPv6 address assigned, but the boot process fails (and no traffic is getting through the firewall).

I was starting to wonder about something: Could it be possible that there is a race condition between the DHCPv6 client and the configuration process? I observed that while the PPPoE connection is being established the output Configuring firewall appears on the console. When 17.1 Beta boots normally (without the modified files), the packet filter rules may get applied before the DHCPv6 client has finished its handshake. If the rules lead to DHCPv6 traffic from the server being blocked, this would explain exactly the behaviour I can observe with Wireshark, namely the client repeatedly sending its Solicitations, but the Advertisements of the server being ignored.
If, on the other hand, the configuration of the packet filter fails (as it did in test 2 above), the DHCPv6 traffic apparently doesn't get filtered and so the DHCPv6 handshake and Prefix Delegation succeed.
Is there a rule that should allow incoming DHCPv6 traffic on the WAN interface?

[fichtner]

Knowing about post-interfaces.lib.inc would be helpful, some changes made.

It could be that the bootup code on 17.1 differs, but then this should be reproducible with 16.7 with the "opnsense-devel" package?

It would help to confirm that PHP 7.0 or FreeBSD 11.0 is not part of the issue :)

[xtof01]

My apologies, I didn't quite understand your last post. What would you like me to try?

Knowing about post-interfaces.lib.inc would be helpful, some changes made.

I tried post-interfaces.lib.inc in test 1 in my last post. Should I do something else with it?

It could be that the bootup code on 17.1 differs, but then this should be reproducible with 16.7 with the "opnsense-devel" package?

Should I install 16.7 and then do a pkg install -f opnsense-devel? Will that require some other adjustments? Should I also try post-interfaces.lib.inc in this setup?

[xtof01]

Hi,

as I wrote at the end of my comment from Wednesday, I am now pretty sure that the problem lies in the packet filter rules.

Right now I am using the following setup:

• 17.1.b with patch 3bbe389
• internal interface configured for tracking of WAN interface

As described before, this setup boots without error messages, but PD and, consequently, assignment of IPv6 addresses on the internal network doesn't work.

With Wireshark I can see the following happening on the WAN interface:
→ DHCPv6 Solicit
← DHCPv6 Advertise
[several seconds pause]
→ DHCPv6 Solicit
← DHCPv6 Advertise

This repeats over and over.
Now I disable the packet filter by checking "Disable all packet filtering" under "Firewall" -> "Settings" -> "Advanced".
And as soon as the Firewall sends its next Solicit message, I see in Wireshark:
→ DHCPv6 Solicit
← DHCPv6 Advertise
→ DHCPv6 Request
← DHCPv6 Reply

And then the DHCPv6 handshake stops repeating on the WAN interface, the internal interface gets an IPv6 prefix assigned and my internal hosts get IPv6 addresses from that prefix (due to the firewall's radvd finally sending Prefix information options in its router advertisements).

Since all this happens the very moment I disable the packet filter rules, it must be that the DHCPv6 responses get blocked on the WAN interface while the packet filter rules are active.

[AdSchellevis]

@xtof01 If I'm not mistaken, there should be rules using port 546/547, can you run the following on your machine and report the output?

grep 546 /tmp/rules.debug

This should return a couple of rules related to DHCPv6.

[xtof01]

@AdSchellevis Thanks for pointing me in the right direction. I found the cause for this problem.
The rules you mention are indeed there:

root@fw:~ # grep 546 /tmp/rules.debug
pass quick on bridge0 inet6 proto udp from {fe80::/10} to {fe80::/10,ff02::/16}  port {546} label "allow access to DHCPv6 server on INTBR" 
pass in quick on bridge0 inet6 proto udp from {fe80::/10} to {(self)}  port {546} label "allow access to DHCPv6 server on INTBR" 
pass quick on pppoe0 proto udp from {fe80::/10}  port {546} to {fe80::/10}  port {546} label "allow dhcpv6 client in WAN" 
pass in quick on pppoe0 proto udp from {any}  port {547} to {any}  port {546} label "allow dhcpv6 client in WAN" 
pass out quick on pppoe0 proto udp from {any}  port {546} to {any}  port {547} label "allow dhcpv6 client in WAN" 
root@fw:~ # 

This looks good so far. However, there are several "block" rules in front of them, in particular:

block in quick on pppoe0 from {<bogons>} to {any} label "block bogon IPv4 networks from WAN" 
block in quick on pppoe0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from WAN" 
block in quick on pppoe0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from WAN" 

The table bogonsv6 is read from a file:

table <bogonsv6> persist file "/usr/local/etc/bogonsv6"

/usr/local/etc/bogonsv6 happens to contain the prefix 8000::/1, which also matches all link-local addresses (fe80::/10). Since link-local addresses are used by DHCPv6 client and server, the incoming DHCPv6 traffic is blocked by the IPv6 bogon rule.
When I disable "Block bogon networks" under "Interfaces" -> "WAN" and reboot the firewall, PD works.

[AdSchellevis]

@xtof01 thanks for analysing this further, it looks like our bogon list is blocked too soon. I will take a look at this one and post a new patch as soon as I have the time.

[AdSchellevis]

@xtof01 can you try this patch 4dbc24e ?

[fichtner]

The command ;)

# opnsense-patch 4dbc24e

[xtof01]

It works! ;-)

I applied the patch, re-enabled "Block bogon networks" on WAN, and rebooted.

While this solution is fine for me, I would like to point out two things I noticed which might not be totally fine for other people.

1) Order of bogon rules wrt. other link-local rules
Here is an excerpt of the current rules.debug file:

[...]
pass quick on pppoe0 proto udp from {fe80::/10}  port {546} to {fe80::/10}  port {546} label "allow dhcpv6 client in WAN" 
pass in quick on pppoe0 proto udp from {any}  port {547} to {any}  port {546} label "allow dhcpv6 client in WAN" 
pass out quick on pppoe0 proto udp from {any}  port {546} to {any}  port {547} label "allow dhcpv6 client in WAN" 
[...]
#block in quick on bridge0 from {<bogons>} to {any} label "block bogon IPv4 networks from INTBR" 
#block in quick on bridge0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from INTBR" 
#block in quick on bridge0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from INTBR" 
[...]
block in quick on pppoe0 from {<bogons>} to {any} label "block bogon IPv4 networks from WAN" 
block in quick on pppoe0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from WAN" 
block in quick on pppoe0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from WAN" 
[...]
pass quick on bridge0 inet6 proto udp from {fe80::/10} to {fe80::/10,ff02::/16}  port {546} label "allow access to DHCPv6 server on INTBR" 
pass quick on bridge0 inet6 proto udp from {fe80::/10} to {ff02::/16}  port {547} label "allow access to DHCPv6 server on INTBR" 
pass quick on bridge0 inet6 proto udp from {ff02::/16} to {fe80::/10}  port {547} label "allow access to DHCPv6 server on INTBR" 
pass in quick on bridge0 inet6 proto udp from {fe80::/10} to {(self)}  port {546} label "allow access to DHCPv6 server on INTBR" 
pass out quick on bridge0 inet6 proto udp from {(self)}  port {547} to {fe80::/10} label "allow access to DHCPv6 server on INTBR" 
[...]

If, for whatever reason, I would enable bogon filtering on my internal interface (bridge0), those rules would probably prevent access to the DHCPv6 server on bridge0, whose rules are below the bogon rules.

2) Distinction between bogons and private networks
The web interface distinguishes between "Block private networks" and "Block bogon networks". For IPv4, I can have the one without the other, i.e. the two sets of addresses are disjoint.
For IPv6, this is not the case, because the IPv6 bogon file, with its 8000::/1 prefix, always includes the private network and link-local addresses. So if I check "Block bogon networks" and uncheck "Block private networks", private IPv6 networks get blocked nonetheless.

[kgleason]

This latest patch resolved my lack of IPv6 connectivity as well. Props to @xtof01 for tracking it down.

[fichtner]

@xtof01 @xtof01 thank you both

I'll leave this open for further bogon feedback. The fix is included in 17.1.r1 which is due tomorrow.