Warning: /etc/resolv.conf is not a symbolic link to /run/resolvconf/resolv.conf

[thomashw]

Distribution (run cat /etc/os-release):
VERSION="19.10"

Issue/Bug Description:
When connecting to my work's VPN using F5's BIG-IP Edge Client for Linux, I saw this warning:
/etc/resolvconf/update.d/libc: Warning: /etc/resolv.conf is not a symbolic link to /run/resolvconf/resolv.conf. The VPN connection was successful, but DNS didn't work. I was able to ping IPs on their network, but could not access any internal sites using their domain names.

Running systemd-resolve --status after making the VPN connection showed my original DNS entries were still being used, not my work's, despite the file at /run/resolvconf/resolv.conf being successfully updated with my work's DNS entries.

Steps to reproduce (if you know):
I would think anything that tries to dynamically update DNS entries via /run/resolvconf/resolv.conf would reproduce this issue.

Expected behavior:
Dynamically updating DNS entries should work.

Other Notes:
After googling, I was able to restore the symbolic link using sudo dpkg-reconfigure resolvconf which fixed the issue.

[jaxn]

I am noticing the same thing while using wg-quick up for my Wireguard VPN.

sudo dpkg-reconfigure resolvconf also fixed it for me.

[hellojakecross]

This is difference behaviour to a standard Ubuntu install.

To get my openconnect VPN connection working with DNS in Ubuntu, I do this:

sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
sudo apt install -y openconnect network-manager-openconnect network-manager-openconnect.gnome

I thought this would be the same in Pop!_OS. Not the case. So when I setup the VPN connection through the gnome integration, it connected fine. DNS was not working.

Ok... I'll try using openconnect from a terminal and see if the network-manager-* part was an issue. The connection was established, but it complained about /run/resolvconf/resolv.conf. That seemed odd to me - this was not the symlink I was required to create for Ubuntu to have working DNS with VPN.

Details of how to get systemd to work with resolv.conf are here: https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html

Is this symlink specific to how Pop!_OS works with systemd? Or is Pop!_OS bypassing the standard systemd setup, meaning the advice given on freedesktop.org isn't relevant?

[funnylookinhat]

sudo dpkg-reconfigure resolvconf fixed it for me as well (I use openconnect)

[turin331]

Same issue on Pop_OS 20.04. Can also validate that
sudo dpkg-reconfigure resolvconf

Fixed the dns issues.

[digitalformula]

Another vote for the above fixes working on Pop OS 20.04.

I'm using OpenConnect version v8.10-10-g58670330, built from source via Ansible.

[dlenski]

Other OpenConnect v8.10 users are affected by this, apparently. https://gitlab.com/openconnect/openconnect/-/issues/178

Should this be considered a bug in Pop OS?

Should OpenConnect's vpnc-script used for routing configuration be detecting Pop OS specifically in some way, and interacting with systemd-resolved in some different way?

[DCdeBrabander]

(Running Pop!_OS 20.04 - 5.4.0-7642-generic )
Using openconnect CLI solution and also had some 'DNS resolving' issues.
Currently still testing but seems sudo dpkg-reconfigure resolvconf did the trick.

[FarisZR]

when is this going to be fixed?

[digitalformula]

Should this be considered a bug in Pop OS?

I think it should be considered a Pop OS bug, yes. I also ran the same version of OpenConnect on Ubuntu for some time and never had to run sudo dpkg-reconfigure resolvconf then.

For what it's worth, I rebuilt my main workstation on Pop OS 20.04 last night and can confirm this issue is still there and that the same fix solves the problem.

[jacobgkau]

The default configuration is that /etc/resolv.conf is a symlink to /run/systemd/resolve/stub-resolv.conf, because Pop!_OS uses systemd-resolved and as explained in the documentation that @eotfofiw linked above:

systemd-resolved maintains the /run/systemd/resolve/stub-resolv.conf file for compatibility with traditional Linux programs. This file may be symlinked from /etc/resolv.conf. This file lists the 127.0.0.53 DNS stub (see above) as the only DNS server. It also contains a list of search domains that are in use by systemd-resolved. The list of search domains is always kept up-to-date... This mode of operation is recommended.

Answering the question from earlier:

Is this symlink specific to how Pop!_OS works with systemd? Or is Pop!_OS bypassing the standard systemd setup, meaning the advice given on freedesktop.org isn't relevant?

The symlink is not specific to Pop!_OS at all. The default configuration is what's recommended in the systemd documentation that you linked.

When you use the GUI to modify your DNS settings (through GNOME Control Center), the systemd configuration is updated appropriately via the APIs that systemd-resolved provides. Some VPN programs (such as OpenVPN) are able to work with systemd-resolved. I'm not sure why OpenConnect doesn't work with this setup, since it appears to support systemd-resolved.

When you run sudo dpkg-reconfigure resolvconf and choose "Yes" in the menu, the symlink to /run/systemd/resolve/stub-resolv.conf is replaced with a symlink to /run/resolvconf/resolv.conf. This puts systemd-resolved into the fourth mode listed in its documentation:

Alternatively, /etc/resolv.conf may be managed by other packages, in which case systemd-resolved will read it for DNS configuration data. In this mode of operation systemd-resolved is consumer rather than provider of this configuration file.

resolvconf is only installed by default because pop-desktop depends on it (I'd be curious whether OpenConnect has the issue with the default stub-resolv.conf symlink if resolvconf isn't installed.) The default symlink to systemd-resolved is created at install time by distinst: https://github.com/pop-os/distinst/blob/2309b998504615b405bc1d530f5b24469f75f116/src/installer/steps/configure/chroot_conf.rs#L274

The symlink used to point to resolvconf. It was changed to systemd-resolved's stub resolver in October 2018: pop-os/distinst#147 Changing the default back to resolvconf would mean reverting that change. Potential downsides might be that some of systemd-resolved's features (like caching and optional DNSSEC validation) don't work. (It does look like resolvconf will point to systemd-resolved by default anyway, but the full consequences would need to be double-checked.)

[faraonc]

Running sudo dpkg-reconfigure resolvconf did the trick for F5 BIG-IP Edge Client.

[gmmarconi]

I'm also confirming that running
sudo dpkg-reconfigure resolvconf and rebooting
fixed my problem for f5fpc vpn CLI on Pop!_OS 20.10

[legioz]

(Running Pop!_OS 20.04 - 5.4.0-7642-generic )
Using openconnect CLI solution and also had some 'DNS resolving' issues.
Currently still testing but seems sudo dpkg-reconfigure resolvconf did the trick.

I had the same problem with this same distro in different machines, maybe is something misconfigured in Pop!_OS too

[nigurr]

This is life saver. I was trying to find out what's going wrong for past 6 hours and figured out that this is indeed mis-configured in Pop OS only.

[FDrebin]

Anyone have a fix aside from running the above command every time? Every time I toggle between home and WireGuard VPN I need to perform the re-configure resolvconf and sometimes I need to reboot after making the change which does not always work well when the VPN is manually triggered.

[konflic]

sudo dpkg-reconfigure resolvconf - worked for me also

[amaurinorato]

This sudo dpkg-reconfigure resolvconf saved my day. I was about to giveup of Pop os

[sclaflin]

Same problem after upgrading to 22.04. sudo dpkg-reconfigure resolvconf and rebooting solved it.

[eSPiYa]

Encountering the same issue on a freshly installed Pop!_OS 22.04. /etc/resolv.conf is a different file from /run/resolvconf/resolv.conf. The former points to 127.0.0.53 with "options edns0 trust-ad" and "search .".

I'm not sure if this is related but the same issue is in ZorinOS 16.1(based on Ubuntu 20.04) and Ubuntu 22.04 server(Raspberry pi 4). Freshly installed.

[rsalunga29]

I'm on Pop!_OS 22.04 LTS

Running sudo dpkg-reconfigure resolvconf solved this issue.

[Tiedye]

From the manpage for resolvconf (which is forwarded to resolvectl on popos)

resolvectl is a multi-call binary. When invoked as "resolvconf" (generally achieved by means of a symbolic link of this name to the resolvectl binary) it is run in a limited resolvconf(8) compatibility mode.

It seems that by default this is not the case (this gave me such a headache figuring that out), is there a "correct" way of creating this link so calls to resolveconf update systemd? @jacobgkau

[jacobgkau]

@Tiedye The workaround everyone else has been using is forcing resolvconf to work correctly and essentially override systemd. If you want to go the other direction and remove resolvconf, then you'd just need to create the binary symlink so running resolvconf actually runs systemd (via resolvectl).

Removing resolvconf is usually not possible since it's a pop-desktop dependency, but I'm seeing if it would be possible to remove that dependency with pop-os/desktop#118. It sounds like at least one person in our Mattermost chat has had success with this so far.

[jacobgkau]

Would anyone with an F5 BIG-IP VPN (@thomashw, @faraonc, @gmmarconi) be able to test if it works with systemd's resolveconf compatibility mode? WireGuard definitely does, so I'm looking to check one of the other affected VPN types. F5's VPN client isn't publicly available (it's deployed from the server, which is also not publicly available.)

The steps to test would be:

• Start while disconnected from the VPN.
• Run sudo apt-manage add popdev:remove-resolvconf
• Run sudo apt update
• Run sudo apt full-upgrade
• Run sudo apt install systemd-resolvconf. (This should remove the regular resolvconf package.)
• Reboot and try connecting to the VPN.
• If it doesn't work, run sudo rm /etc/resolvconf/interface-order and then try connecting again. (It's known that WireGuard requires this step.)

[gene1wood]

As a data point, I've observed this problem in PopOS 22.04, likely being triggered by https://github.com/netbirdio/netbird which I run.

[jacobgkau]

@gene1wood This issue is closed because the problem was fixed as far as we could tell.

Can you please provide more information about the issue you're seeing with netbird? What symptom(s) are you experiencing that pointed you towards this issue? If you manually install resolvconf (sudo apt install resolvconf) and reboot, do you still have an issue? When did you first install Pop!_OS on the system you're using?

[gene1wood]

How that pop-os/desktop#118 has been merged, what is the way to get this behavior into an existing PopOS 22.04 installation? Should I uninstall the resolvconf package and install the systemd-resolvconf package?

[jacobgkau]

@gene1wood If you installed your system prior to ~June 2023, then you can use one of the two workarounds listed in one of my posts there:

On existing installations, this won't change anything by default. Users affected by issues with the old configuration have two options:

• Run sudo dpkg-reconfigure resolvconf and select Yes to make the /etc/resolv.conf symlink point to /run/resolvconf/resolv.conf. (This workaround has already been available.)
• Run sudo apt install systemd-resolveconf and, at least for some apps, sudo rm /etc/resolvconf/interface-order.

[gene1wood]

@jacobgkau Sure thing.

Can you please provide more information about the issue you're seeing with netbird?

Sure, starting in the last week or so, I began experiencing DNS resolution stopping working. It's happened 5 or so times in the last week.

I'm assuming this is an interaction between netbird and resolve.conf

Of note though, I've been using this PopOS installation for a year without this problem and I've had Netbird installed and running that whole time.

What symptom(s) are you experiencing that pointed you towards this issue?

I found this log line which brought me to this issue

May 20 14:02:07 wendy sh[67843]: /etc/resolvconf/update.d/libc: Warning: /etc/resolv.conf is not a symbolic link to /run/resolvconf/resolv.conf

If you manually install resolvconf (sudo apt install resolvconf) and reboot, do you still have an issue?

I haven't tried it yet because, what I've done each time the problem has happened is rebooted and then the problem went away (at least for a bit)

When did you first install Pop!_OS on the system you're using?

I'm running a System76 Laptop that came with Pop!_OS 22.04 installed on it on 2023-06-08.

If you installed your system prior to ~June 2023, then you can use one of the two workarounds listed in one of my posts there:

Thank you, my system is from right around June 2023, so maybe I'm affected by this.

I'll run the sudo dpkg-reconfigure resolvconf and select Yes and see if that fixes it.

Thanks again!