IPv6 not working when using 6rd with a /64 prefix

[paride]

Describe the bug

All what follows is on OPNsense 20.1.4 (i386/OpenSSL).

My ISP supports IPv6 using 6rd but sadly only assigns a single /64. However this should be enough to have radvd/SLAAC on the LAN interface, and indeed it works fine with the router provided by the ISP. But let's proceed by steps.

I can properly set up 6rd on the WAN interface. I need to manually add wan_stf as the IPv6 default route to workaround #3903, but then the tunnel works fine:

root@net6501:~ # ifconfig wan_stf
wan_stf: flags=4041 metric 0 mtu 1280
	inet6 2001:b07:5d2e:58c7:: prefixlen 64 
	nd6 options=1
	v4net 93.46.88.199/0 -> tv4br 81.208.50.214
	groups: stf 

root@net6501:~ # route -6 get default
   route to: default
destination: default
       mask: default
        fib: 0
  interface: wan_stf
      flags: 
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1280         1         0 

root@net6501:~ # ping6 -c1 ipv6.google.com
PING6(56=40+8+8 bytes) 2001:b07:5d2e:58c7:: --> 2a00:1450:4002:805::200e
16 bytes from 2a00:1450:4002:805::200e, icmp_seq=0 hlim=57 time=7.416 ms

The LAN interface is configured to track the WAN interface, with IPv6 prefix ID 0x0 (IIUC as I have a single /64 this is the only choice). The interface configuration looks correct: the LAN interface get assigned the :1 address of the 6rd /64:

bridge0: flags=8843 metric 0 mtu 1500
	ether 02:9d:ee:f2:88:00
	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 
	inet6 2001:b07:5d2e:58c7::1 prefixlen 64 
	inet6 fe80::29d:eeff:fef2:8800%bridge0 prefixlen 64 scopeid 0xa 
	nd6 options=1
	groups: bridge 
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: ath0_wlan1 flags=143
	        ifmaxaddr 0 port 9 priority 128 path cost 33333
	member: em1 flags=143
	        ifmaxaddr 0 port 2 priority 128 path cost 55

and the router advertisement also works, as my laptop on the LAN gets assigned addresses on the right /64:

3: wlp61s0:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 48:f1:7f:b4:3e:9e brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.138/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp61s0
       valid_lft 4929sec preferred_lft 4929sec
    inet6 2001:b07:5d2e:58c7::2000/128 scope global dynamic noprefixroute 
       valid_lft 4932sec preferred_lft 2232sec
    inet6 2001:b07:5d2e:58c7:84b2:3297:aa4c:135a/64 scope global temporary dynamic 
       valid_lft 86267sec preferred_lft 14267sec
    inet6 2001:b07:5d2e:58c7:f1c5:2ef7:e567:afd2/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 86267sec preferred_lft 14267sec
    inet6 fe80::4470:3f96:5d7c:54d0/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

However I can't ping the router (2001:b07:5d2e:58c7::1) from the laptop: there is no reply to ping. This is pretty strange as there are two interfaces with IP addresses on the same network and they communicate.

I am specifically blaming the fact that the 6rd assigned network is a /64 because if I misconfigure it as a /63 then router and laptop can communicate with the addresses assigned by radvd, but of course the 6rd tunnel does not work.

This is somehow similar to #2663, but a lot of things changed since OPNsense 18.7. I don't know if this ever worked with OPNsense, but it is reported working with pfSense.

[paride]

If I remove the inet6 address from the wan_stf interface like this:

ifconfig wan_stf inet6 2001:b07:5d2e:58c7:: -alias

the router <-> laptop communication starts working immediately (I get ping replies), so I think the wan_stf interface is interfering with the traffic what should go to the LAN interface. Maybe traffic for the LAN interface gets incorrectly routed to wan_stf?

Needless to say that that the 6rd tunnel stops working when I do this, but at least we're sure that on the LAN side everything is configured correctly.

[tamersaadeh]

I created a script in /usr/local/etc/rc.syshook.d/start/ which automates this after a reboot, it also gives the wan_stf a ULA which you can use to ping the router from other devices, however the router cannot access the internet unless you explicitly use the LAN interface/address

script 55-fix-6rd (must be executable):

#!/bin/sh

echo "+------------------------------+"
echo "|                              |"
echo "| Fixing 6rd.................. |"
echo "|                              |"
echo "+------------------------------+"

BAD_IP="$(/sbin/ifconfig wan_stf | /usr/bin/sed -n '/.inet6 /{s///;s/ .*//;p;}')"
RANDOM_ULA="$(echo fd`openssl rand -hex 7`: | /usr/bin/sed 's/.\{4\}/&:/g')"
/sbin/route add -inet6 default -interface wan_stf
/sbin/ifconfig wan_stf inet6 $BAD_IP -alias
/sbin/ifconfig wan_stf inet6 $RANDOM_ULA/64 alias

echo "+------------------------------+"
echo "|                              |" 
echo "| Done fixing 6rd............. |"
echo "|                              |"
echo "+------------------------------+"

Hope this helps and maybe Fastweb will give us more than just a single /64 some day. 😉

[paride]

Thanks, that's an interesting workaround. Way better than nothing, however there has to be a better way, as it works flawlessly with pfSense 2.3.5 (the latest version supporting i386 hardware). I couldn't test with newer releases but it's not reported as broken AFAIK.

[ja-finn]

it works flawlessly with pfSense 2.3.5 (the latest version supporting i386 hardware). I couldn't test with newer releases but it's not reported as broken AFAIK.

Can confirm that it's plug and play on 2.4.5 as well, no issues.

[fichtner]

I will close this issue if off-topic discussions continue to take place. Discuss OPNsense solutions or troubleshooting or use whatever works for you.

[paride]

@fichtner thanks for chiming in. Please allow me to disagree: I think we have a bug here, and not a support request, and I think mentioning other solutions using the same underlying operating system but behaving differently is OPNsense troubleshooting. "Looking at differences" is probably the single most powerful debugging technique have, and is the basis of more advanced ones as bisecting, A/B testing and so on.

If we are here is because we have reasons to prefer OPNsense to other solutions, and are willing to allocate some time to try to make it better.

This said: yes, let's stay on topic. :)

[fichtner]

I do not disagree that it may be a bug. I spent a lot of work to make this happen remotely two years ago. The issues that I see:

• No way for a dev to test such a setup natively so we did everything remotely with brave testers.
• No patches for this in FreeBSD after all these years (not just the last two), which in my opinion is done deliberately.

I'm happy to look into it, but we run with a lot of restrictions and need help to point us to the line of code that needs to be changed.

I do not believe tracking a /64 prefix through a router is going to work and prone to breakage as far as IPv6 connectivity on the router itself goes.

Cheers,
Franco

[fichtner]

PS: I checked the code and I can see nothing obvious except that we're flying blind here with not a single log message attached, e.g.

https://github.com/opnsense/core/blob/master/src/etc/inc/interfaces.inc#L2688

That would be an easy thing to compare...

[fichtner]

Question: if you stop tracking LAN and just add a static ULA in the configuration, then LAN should receive advertisements and if you're in luck you can reach WAN?

[paride]

@fichtner I can't take down my connection at the moment, I can try over the weekend. However in my (possibly flawed) understanding ULA addressed (fd00::/8) are not meant to the routed or routable to the IPv6 Internet:

[...] Its purpose in IPv6 is analogous to IPv4 private network addressing. Unique local addresses may be used freely, without centralized registration, inside a single site or organization or spanning a limited number of sites or organizations. They are routable only within the scope of such private networks, but not in the global IPv6 Internet.

[fichtner]

It looks like you are right. It works for routing and internal communication but not to reach the Internet from the client.

Looking at the workaround presented it forces the IPv6 traffic out the WAN interface, which means it "solves" the issue of the routing table dilemma if it should send to WAN or LAN, but it would probably also break reaching your LAN devices from the firewall itself.

If that is the approach employed by the other firewall (I am only guessing) it's not so much desirable. You can try to comment out the following, but it might need further manipulation of the system still:

core/src/etc/inc/system.inc

Line 458 in 50f0b15

$realif = null;

The thing is this isn't portable and related to /64 usage indeed and has other implications such as not being able to delegate more than one subnet internally or may conflict with Multi-WAN setups.

[maurice-w]

Changig the prefix length on the wan_stf interface to /128 might allow the router as well as hosts in the LAN to access the Internet. But I can't easily test this.

ifconfig wan_stf inet6 $BAD_IP -alias
ifconfig wan_stf inet6 $BAD_IP/128

[AdSchellevis]

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.