OpenVPN Connection Status - Unable to Contact Daemon

[pricklydevil]

Posted this on the forum but no acknowledgement .. hopefully here will help

I've got several OVPN connections established and working (confirmed by connections to WAN via 4G and from LAN); however, the dashboard and menu option (VPN > OpenVPN > Connection Status) are all giving "Unable to contact daemon Service not running?" even though they're definitely working.

Reboots are not solving the issue, nor killing the processes.

I've attached logs from the OVPN service as well as graphical evidence.

PS AUX output
root@router:~ # ps aux | grep openvpn root 23048 0.0 0.1 1085760 6728 - Ss Thu13 0:02.57 /usr/local/sbin/openvpn --config /var/e root 24452 0.0 0.1 1085760 6924 - Ss Thu13 2:31.13 /usr/local/sbin/openvpn --config /var/e root 30568 0.0 0.1 1085760 6888 - Ss Thu13 2:30.40 /usr/local/sbin/openvpn --config /var/e root 32392 0.0 0.1 1085760 6920 - Ss Thu13 2:56.28 /usr/local/sbin/openvpn --config /var/e root 86435 0.0 0.0 1080528 2840 0 S+ 18:50 0:00.00 grep openvpn

openvpn.log

[csriebel08]

This is a problem for me as well only the openvpn client drops the tunnel and is not working at all. If I start the service through the gui then it will start and establish a connection but then the daemon stops and the tunnel goes down. If you can help me out I can provide logs.

[fichtner]

Sounds like this one? #1931

[csriebel08]

@fichtner I wish my situation was like this but my WAN port and connection is fine. This for me started a while ago in version 17 however I didn't think it was a big deal as I could start the service and my VPN client would stay running and the tunnel would stay up. However after moving to 18.7 the current version I have found this behavior as described with the OP with the exception that as far as I can tell my vpn is not working (that is why I offered to gather logs to help but I don't where to start that would help out). This behavior is to the point that my VPN client will not stay running at all and in the GUI I get what is described above. I would note that my VPN Server has no issues and is working fine. I also would mention that the only other behavior that I noticed with this is that the NTP service seems to shut down when starting the VPN client service. So this is what happens:

1. Start the firewall everything comes up fine except for the OpenVPNclient service is stopped and the client instance statistics shows: "Uncalbe to contract daemon Service not running?" and the interface for this shows down.
2. Start the OpenVPNClient service, this starts and does bring up the tunnel, interface ect. but when this happens the NTP service goes offline.
3. Start the NTP service and the OpenVPNClient service stops
4. NTP service remains running

I looked at #1931 and the OP in this talks about his WAN not being stable and that might be the reason for the issues, in my case this is not an issue. The only thing that I found in that post is that visualstation mentions his NTP service however I don't know if it is similar to my situation as he posted one time then was dismissed because he did not align with the OP in behavior.

[pricklydevil]

I'd like to say my WAN is unstable, but it's not. Solid uptime with no maintenance work being carried out (confirmed by the ISP and confirmed that the cable modem has been solid for the past 20 days straight). Rules out #1931.

The dashboard is attempting to locate PIDs that are not valid, therefore throwing the error of unable to connect to daemon.

Slightly more info: So I created a new OpenVPN connection to my provider on a completely different server. Connection was established, however the connection status page does display a connected since date but the rest is 0 and the status is "connecting"

[csriebel08]

I had to revert to version 17.7.5, I continuously was having my vpn client shut down and could not keep it up. I also had a problem where it would not let me add any hosts to my aliases or add new aliases for that matter.

[fichtner]

All past, present and future disappointment aside... the code tries to call stream_socket_client() on the unix socket that is supposed to be created by OpenVPN, but is refused by the file system. let's see what the actual error is with 0a8794f:

# opnsense-patch 0a8794f

Please send a screenshot of the status page with the additional error reporting in place.

[fichtner]

I found this note, maybe it is relevant:

"You need to bind each OpenVPN client to enable its management daemon: use 'Local port' setting in the OpenVPN client screen"

[pricklydevil]

Updated screenshot after apply patching 0a8794f

I've also attempted to set the local port using 0, rather than blank, for a random port. Still the same error. I've also set a local port in the high 20000's, away from any other port that is currently open, same result.

[fichtner]

Ok, let's see if the socket is there or not...

# ls -lah /var/etc/openvpn/*.sock

What kind of clients are these? tun/tap, p2p?

[pricklydevil]

root@router:~ # ls -lah /var/etc/openvpn/*.sock
srwxrwxrwx 1 root wheel 0B Aug 9 18:08 /var/etc/openvpn/client2.sock
srwxrwxrwx 1 root wheel 0B Aug 9 18:04 /var/etc/openvpn/client3.sock
srwxrwxrwx 1 root wheel 0B Aug 9 18:05 /var/etc/openvpn/client4.sock
srwxrwxrwx 1 root wheel 0B Aug 9 18:05 /var/etc/openvpn/client5.sock
srwxrwxrwx 1 root wheel 0B Aug 2 13:15 /var/etc/openvpn/server1.sock

All four are TUN adapters, 3 running as clients to connect to remote servers and 1 which is running as a server for inbound connections

[pricklydevil]

Recreated all of these today, still exactly the same error on the dashboard and connection status

[fichtner]

@pricklydevil "connection refused" seems to point to two possibilities: either OpenVPN is refusing to accept a management connection which would show in the OpenVPN. Or it fails to reclaim the socket and this should also be in the log. To rule out the latter let's remove the socket before starting OpenVPN:

# opnsense-patch 2916d62

[pricklydevil]

@fichtner Okay so patch applied, cleared VPN log, took down one of the VPN connections and brought it back online.

Dashboard still shows Connection Refused (61), with the logs showing below

Aug 13 19:28:01 | openvpn[29779]: UDPv4 link remote: [AF_INET]XXX.XXX.XXX.XXX:12200
Aug 13 19:28:01 | openvpn[29779]: UDPv4 link local (bound): [AF_INET]XXX.XXX.XXX.XXX:0
Aug 13 19:28:01 | openvpn[29779]: Socket Buffers: R=[42080->42080] S=[57344->57344]
Aug 13 19:28:01 | openvpn[29779]: TCP/UDP: Preserving recently used remote address: [AF_INET]31.3.153.248:12200
Aug 13 19:28:01 | openvpn[29779]: Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
Aug 13 19:28:01 | openvpn[29779]: Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
Aug 13 19:28:01 | openvpn[29779]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Aug 13 19:28:01 | openvpn[29779]: MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client5.sock
Aug 13 19:28:01 | openvpn[29326]: library versions: LibreSSL 2.6.5, LZO 2.10
Aug 13 19:28:01 | openvpn[29326]: OpenVPN 2.4.6 amd64-portbld-freebsd11.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jul 28 2018
Aug 13 19:28:01 | openvpn[29326]: WARNING: file '/var/etc/openvpn/client5.up' is group or others accessible
Aug 13 19:27:54 | openvpn[18898]: SIGTERM[hard,] received, process exiting
Aug 13 19:27:54 | openvpn[18898]: event_wait : Interrupted system call (code=4)`

[pricklydevil]

@fichtner Update: rebooted today after applying the 18.7.1 update and now everything is showing properly on the dashboard and VPN connection status.

Really not sure what has caused it though but would like to help and figure out why the problem existed and if one of the code fixes you pushed helped highlight the cause.

[fichtner]

@pricklydevil not sure either, but happy this helped. what I saw was that normally OpenVPN removes the sockets on its own. It could have been related to a power failure leaving damaged socket files on the disk? I'll push that cleanup to 18.7.2 since we don't know more and it certainly doesn't cause any side effects. If you are satisfied please close the ticket. :)

[pricklydevil]

Definitely happy 👍 No power failures as I know of as everything is attached to a UPS but it's still definitely possible.

Thanks for all your help @fichtner I really do appreciate it