Wireguard | Lost peer allowed ips config after upgrade 23.7.6

[shtirlic]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

After performing latest upgrade and restart I lost wg connectivity and noticed that via
wg show I have allowed ips : (none) while my /usr/local/etc/wireguard/wg1.conf is perfectly fine.

To Reproduce

Steps to reproduce the behavior:

1. Upgrade to 23.7.6

Expected behavior
Should be working fine after upgrade

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 23.7.6

[fichtner]

Can you confirm that the settings were not dropped from the config.xml? Did you reboot? Which was the version being used before?

[shtirlic]

Yes i rebooted after the upgrade from 23.7.5, i see the correct data in config.xml see:

   <wireguard>
      <general version="0.0.1">
        <enabled>1</enabled>
      </general>
      <server version="0.0.4">
        <servers>
          <server uuid="118828b5-abbd-45e5-9c06-d150bff4b9d5">
            <enabled>1</enabled>
            <name>OPSE_WG</name>
            <instance>1</instance>
            <pubkey></pubkey>
            <privkey></privkey>
            <port/>
            <mtu/>
            <dns/>
            <tunneladdress>10.0.35.210/32</tunneladdress>
            <disableroutes>0</disableroutes>
            <gateway/>
            <peers>2c0e83ef-a352-4536-9f78-b5de64de3576</peers>
          </server>
        </servers>
      </server>
      <client version="0.0.7">
        <clients>
          <client uuid="2c0e83ef-a352-4536-9f78-b5de64de3576">
            <enabled>1</enabled>
            <name>m35</name>
            <pubkey></pubkey>
            <psk/>
            <tunneladdress>10.0.35.0/24,10.0.30.0/24,10.0.110.0/24</tunneladdress>
            <serveraddress></serveraddress>
            <serverport>3478</serverport>
            <keepalive>25</keepalive>
          </client>
        </clients>
      </client>
    </wireguard>

My bet it's because some renaming was introduced from clients -> peers ?

[fichtner]

The renaming should only affect the GUI. If it's still displayed then it's fine. wg.conf creation in particular has not been changed. Does it work if you manually restart it?

[shtirlic]

OK, I manually restarted via UI and it comes online with proper wg show allowed peers, then I rebooted the instance and it again broken

[fichtner]

It may be related to this change then 70df688a9 particularly src/opnsense/scripts/Wireguard/wg-service-control.php

[fichtner]

Do you have virtual IPs set for this one or assigned the wg interface and added an IPv4 configuration?

[shtirlic]

Yes, I have wg intreface and it's set ipv4

[fichtner]

mullvad I supposed? Then same issue as described here: https://forum.opnsense.org/index.php?topic=36403.0

[shtirlic]

No mullvad, just my remote wg server, I deleted ipv4 static config for WG interface and after the reboot wg is working without manual restart.

[fichtner]

Ok, sicne 23.7.6 configuring an IPv4 or IPv6 mode is not possible anymore. Then I'm switching this back to "support" and I hope we can close? :)

[fichtner]

My colleague says you should change:

<tunneladdress>10.0.35.210/32</tunneladdress>

to:

<tunneladdress>10.0.35.210/24</tunneladdress>

Cheers,
Franco

[shtirlic]

Maybe some workaround?

The following input errors were detected:

Cannot assign an IP configuration type to a tunnel interface.

Now i got this.

[shtirlic]

As I understand since there is no ipv4 config for WG intreface, I should update firewall rules and other settings to allow things to work? For example UI is not accessible via WG ip, before with ipv4 static ip it was working fine.

[fichtner]

That's probably #6934 (comment) (no other changes necessary as far as I can tell)

[shtirlic]

Did the change already to /24, via tcpdump I got

06:58:43.807944 IP 10.0.35.1.54222 > 10.0.35.210.443: Flags [S], seq 3653867588, win 65535, options [mss 1380,sackOK,TS val 3343312213 ecr 0,nop,wscale 9], length 0
06:58:44.831061 IP 10.0.35.1.54222 > 10.0.35.210.443: Flags [S], seq 3653867588, win 65535, options [mss 1380,sackOK,TS val 3343313236 ecr 0,nop,wscale 9], length 0
06:58:46.847131 IP 10.0.35.1.54222 > 10.0.35.210.443: Flags [S], seq 3653867588, win 65535, options [mss 1380,sackOK,TS val 3343315252 ecr 0,nop,wscale 9], length 0
06:58:51.011364 IP 10.0.35.1.54222 > 10.0.35.210.443: Flags [S], seq 3653867588, win 65535, options [mss 1380,sackOK,TS val 3343319412 ecr 0,nop,wscale 9], length 0

with no replies,

but no, ssh, https acces, icmp is working.

That's the reason I added ipv4 static ip for WG at first place, without it, nothing going trough from the WG server

[fichtner]

Can you post the ifconfig output of wg1 or send it privately to franco AT opnsense DOT org ?

Thanks,
Franco

[shtirlic]

wg1: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
        options=80000<LINKSTATE>
        inet 10.0.35.210 netmask 0xffffff00
        groups: wg wireguard
        nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD>

[fichtner]

It looks correct. Try to reload the firewall rules:

# configctl filter reload

[shtirlic]

I see something strange here
sockstat -4 -l

USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
unbound  unbound    381   5  udp4   10.0.10.1:53          *:*
unbound  unbound    381   6  tcp4   10.0.10.1:53          *:*
unbound  unbound    381   7  udp4   10.0.35.210:53        *:*
unbound  unbound    381   8  tcp4   10.0.35.210:53        *:*
unbound  unbound    381   9  udp4   127.0.0.1:53          *:*
unbound  unbound    381   10 tcp4   127.0.0.1:53          *:*
unbound  unbound    381   11 udp4   10.0.10.1:53          *:*
unbound  unbound    381   12 tcp4   10.0.10.1:53          *:*
unbound  unbound    381   13 udp4   10.0.35.210:53        *:*
unbound  unbound    381   14 tcp4   10.0.35.210:53        *:*
unbound  unbound    381   15 udp4   127.0.0.1:53          *:*
unbound  unbound    381   16 tcp4   127.0.0.1:53          *:*
unbound  unbound    381   17 tcp4   127.0.0.1:953         *:*
root     ntpd       62781 21 udp4   *:123                 *:*
root     ntpd       62781 22 udp4   10.0.110.20:123       *:*
root     ntpd       62781 23 udp4   10.0.10.1:123         *:*
root     ntpd       62781 26 udp4   127.0.0.1:123         *:*
root     ntpd       62781 27 udp4   10.0.35.210:123       *:*
root     lighttpd   21366 4  tcp4   127.0.0.1:43580       *:*
root     miniupnpd  9309  8  tcp4   *:2189                *:*
root     miniupnpd  9309  9  udp4   *:1900                *:*
root     miniupnpd  9309  10 udp4   10.0.10.1:24625       *:*
root     miniupnpd  9309  12 udp4   10.0.10.1:5351        *:*
dhcpd    dhcpd      67064 10 udp4   *:67                  *:*
root     lighttpd   58722 7  tcp4   127.0.0.1:443         *:*
root     lighttpd   58722 9  tcp4   10.0.10.1:443         *:*
root     lighttpd   58722 10 tcp4   127.0.0.1:80          *:*
root     lighttpd   58722 12 tcp4   10.0.10.1:80          *:*
root     sshd       55459 4  tcp4   127.0.0.1:22          *:*
root     sshd       55459 5  tcp4   10.0.10.1:22          *:*
?        ?          ?     ?  udp4   *:11524               *:*

there are olny unbound and ntpd running open on WG ip, no sshd or lighttpd
Maybe it's related since there is no ipv4 config for interface ?

Update:
I restarted sshd service and it's got WG ip and started listen on this port

[fichtner]

inet 10.0.35.210 netmask 0xffffff00 is the IPv4 config... but the reload hook may be missing. You are looking for web GUI?

# configctl webgui restart

[shtirlic]

Yes, configctl webgui restart helps and lighttpd started on the WG ip, so what should I do make it reloaded after reboot?
I guess the services started before wg tunnel don't know the IP to bind and after WG is started and IP is known there is no reload hook for webgui/sshd .

[fichtner]

To be frank the listening behaviour and its problems is explained here:

https://docs.opnsense.org/manual/settingsmenu.html#listen-interfaces

The behaviour depends on a number of factors... I think a static WAN setup might not do what you want in this case. How is your WAN IPv4/IPv6 mode configured?

[shtirlic]

Thank you for pointing to the right place in docs, since from now there is no ipv4 config possible for wg, I will switch to listen on all interface for webgui and ssh and make firewall do the things. I tested it now and it works perfectly.

The interface must always be available, so do not try to bind to vpn instances of any kind (OpenVPN, Wireguard, 

[fichtner]

Ok, close then? :)