IPv6/IPv6 Scripts

[mannkind]

Is IPv6 currently working on OPNSense? I have my WAN set w/DHCP6, but I don't see it running.

When trying to start it manually, I found /var/etc/dhcp6c_wan.conf references /var/etc/dhcp6c_wan_script.sh which tries to start /usr/local/etc/rc.newwanipv6 via fcgicli

[jschellevis]

issue is related to the fact that fcgicli is still trying to use php-fpm and fpm is not running... WIP

[jschellevis]

some changes have been made, please test again with latest git or 15.1.6 when released.

[mannkind]

I've updated to 15.1.6.1 –

After running /var/etc/dhcp6c_wan_script.sh (which just runs /usr/local/etc/rc.newwanipv6 vmx1) manually, I don't get and IPv6 address and all I can see in the logs is as follows…

• opnsense: /usr/local/etc/rc.newwanipv6: rc.newwanipv6: Informational is starting vmx1.
• opnsense: /usr/local/etc/rc.newwanipv6: rc.newwanipv6: Failed to update WAN[wan] IPv6, restarting…

Let me know if there's anything else I can provide/configure/etc.

[pdf]

@mannkind /var/etc/dhcp6c_wan_script.sh doesn't actually start dhcp6c, it's called by it. The dhcp6c client appears to be started by /var/etc/rtsold_${interface}_script.sh, which is in turn called by rtsold.

With that said, @jschellevis I came here to report that rtsold is extremely flakey on boot - it almost never initializes the dhcp6c client. I've found that by changing rtsold from one-shot to daemon mode in interfaces.inc this is very much more reliable.

https://github.com/opnsense/core/blob/master/src/etc/inc/interfaces.inc#L3654

Change mwexec("/usr/sbin/rtsold -1 -p ... to mwexec("/usr/sbin/rtsold -p ...

[jschellevis]

@mannkind @pdf thanks for your feedback. @pdf do you think that running rtsold in deamon mode will solve the entire issue or is more investigation needed?

[pdf]

@jschellevis I can't be certain what @mannkind's issue is, however dhcp6c was certainly not being started reliably on my system so it seems likely to be related. I'm not using DHCPv6 to obtain an address in my setup though, just a PD. There may indeed be other issues with php-fpm further down the chain as you've suggested above.

I suppose it would be nice to know why rtsold was being run in one-shot mode so we can be certain there are no unwanted side-effects, but for my configuration I haven't seen any. The problem may be some sort of timing issue - I never saw dhcp6c started successfully on boot with rtsold in one-shot mode, but running it after the system was fully booted would result in dhcp6c being started correctly. It may be that rtsold needs an address that isn't tentative (once it's passed duplicate detection), which we could possibly wait for using a loop, but that'd block the rest of the boot process. However, running rtsold in daemon mode always reliably starts dhcp6c.

---

Now, with that out of the way... I should probably mention that I'm new to FreeBSD (well, newly returned I guess, haven't worked with it in nearly 15 years), but my reading of how configuration would normally be done (outside of OPNsense) for FreeBSD 9+ is that the functionality that rtsol used to provide would now be handled directly by the kernel.

I've got other issues with OPNsense's IPv6 support, for example I'm meant to receive a dynamic /64 for the wan interface - and the appropriate routes to go with it - via RA, and also receive a prefix via DHCPv6-PD for the LAN side. This is currently impossible with OPNsense, and while it's still possible to route traffic over the WAN via link-local, this should be a supported configuration.

• First, I'd propose dropping rtsol altogether and spawning dhcp6c explicitly for DHCPv6 interfaces (it's already spawned explicitly for SLAAC interfaces, though that should probably also be tunable, because DNS might be expected via RDNSS, handled by openresolv/resolvconf).
• If either SLAAC or DHCPv6 is selected on the WAN, set sysctl tunables net.inet6.ip6.rfc6204w3 and net.inet6.ip6.no_radr to 1 - this will allow accepting RA messages even when forwarding is enabled, and disable accepting route advertisements on all interfaces by default (necessary to avoid upstream routes landing on the wrong interface). With the default for no_radr set to 1, the WAN interface will need to have the flag inverted (inet6 -no_radr) so that routes will be accepted on that interface.
• With rtsol gone, let the kernel handle handle RA messages by setting the appropriate interface flags. For SLAAC interfaces, or for DHCPv6 interfaces with dhcp6prefixonly enabled, add the inet6 accept_rtadv flag. The default for all interfaces should probably be to disable autoconf by setting net.inet6.ip6.accept_rtadv to 0, so that only interfaces specifically set to accept autoconf actually do so.

I think that should get things into better shape, but I'm just going by the documentation - the only box I have to test on right now is pushing real data.

Something to consider here though is interfaces that may not be reliable, like ppp/pppoe/l2tp, etc. It needs to be ensured that these interfaces are properly (re-)configured when they come back up should they fail. Another major problem I have with OPNsense is that mpd5 never seems to recover from a link failure, which means ppp/pppoe connections need to have someone on-site to recover from problems. This may actually be a show stopper for some deployments I was considering, but the reliability component should be a separate issue.

Sorry for the long-winded response, hope some of this makes sense.

[mannkind]

thanks for all the info @pdf and @jschellevis. My current work-around to working IPv6 is to modify /usr/local/etc/inc/interfaces.inc as follows…

• Add -ifdisabled to mwexec("/sbin/ifconfig {$wanif} inet6 accept_rtadv"); – rtsold -D recently started reporting "vmx1 is disabled"… Not sure if that's due to a setting change I made or because of the 15.1.7 update
• Set net.inet6.ip6.rfc6204w to 1 via set_single_sysctl (Or System Tunables) – the only way I could ping6 anything after getting an IPv6 address; found via a bug report in pfsense 2.2
• Remove the -1 from the mwexec("/usr/sbin/rtsold -1 -p ... – as suggested by @pdf

[jschellevis]

@mannkind l'll merge your suggested fixes into the upcoming release 15.1.8.
@pdf thanks for your alternative solution, I will copy that into an enchancement request as I do not have time to test and verify it now. As for the mpd5 issue, can you create a bug report for that with a test scenario I can reproduce?

I'll close this issue shortly with a bugfix that includes all three suggestions.

[fichtner]

Need to reopen this due to the fact that the removal of -1 clobbers /etc/resolv.conf via rtsold(8)'s embedded call to resolvconf(8). Your DNS setup will break. Hybrid setups are broken in any case. Maybe we can route the IPs through /usr/local/etc/rc.resolv_conf_generate instead?

[mannkind]

I guess you know why rtsold was being run in one-shot mode. Might be useful to make a comment in the code :)

For fun, I added the "-1" back to the rtsold in /usr/local/etc/inc/interfaces.inc and rebooted OPNSense. It appears that my IPv6 connectivity still works, and that rtsold is not running. Not sure about the resolv.conf issue as my DNS has been fine, but it was last modified after the reboot.

[pdf]

@mannkind if you don't require DHCPv6 for your IPv6 connectivity, one-shot mode should work, otherwise probably not.

@fichtner I haven't seen this behaviour, can you explain what conditions are required for DNS to be broken?

[fichtner]

@pdf WAN has "DHCP6" and LAN has "Track Interface" setting as IPv6 configuration type.

[pdf]

@fichtner right, and dhclient doesn't start reliably with rtsold running in one-shot mode, which was the original subject of this issue..

[fichtner]

timeout