Ubuntu 17.04 DNS issues after installing

[Meverhart]

Using Ubuntu 17.04. After installing Valet I seem to be having intermittent DNS issues including some common sites (facebook, etc...) not always resolving correctly. I believe this may be related to dnsmasq. Is this a known issue?

[cpriego]

Hi, @Meverhart the intermittent DNS issue isn't normal. Could you please share your /etc/NetworkManager/NetworkManager.conf content?

[Meverhart]

[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=false

[cpriego]

Hmmm, it looks as it should. Ok, let's try changing the control of dnsmasq to NetworkManager. Please change the contents of /etc/NetworkManager/NetworkManager.conf to this:

[main]
plugins=ifupdown,keyfile
dns=dnsmasq

[ifupdown]
managed=false

Copy the file in /etc/dnsmasq.d/valet to /etc/NetworkManager/dnsmasq.d/valet and finally stop the dnsmasq service and restart the network manager service

sudo service dnsmasq stop
sudo service network-manager restart

If you lose your internet connection, simply reconnect.

[Meverhart]

The /etc/dnsmasq.d/valet directory doesn't exist. What file should be there?

[cpriego]

17.04 is not officially released yet so there might lie the issue. The only thing left is try removing the stable branch and installing the multiDistro branch of valet. It uses DnsMasq throught NetworkManager. See if it solves the problems you're having.

[cpriego]

To install the multiDistro branch that is working in the 16.10 version
composer global require "cpriego/valet-ubuntu:dev-multiDistro"

[Meverhart]

Still a multitude of problems whenever dnsmasq is running. Upon stopping the dnsmasq service all problems cease.

[cpriego]

I was able to install Ubuntu 17.04 but clearly the problem is in the development packages. It seems Zesty is using systemd-resolv instead of dnsmasq and network manager no longer manages dns resolution (as opposed to all of the other distros out there right now).

I'm afraid we'll just have to wait until Zesty is officially released.

[javi-dev]

Hi, I'm on 16.10 and I'm noticing the same kind of problems. My /etc/NetworkManager/NetworkManager.conf is

plugins=ifupdown,keyfile,ofono
dns=dnsmasq

[ifupdown]
managed=false

I haven't done any tests or fixes myself yet, I'd probably make things worse. I'm just reporting the issue for now and wondering if this is expected, if it happens to anyone else, and if I should have installed LTS (it's a new Ubuntu installation). I'll provide info about my system if it helps.

Thanks!

Edit: After a bit of research, according to this, taken from this forum thread, 16.10 has the same issue for the same reason, and users are experiencing the same kind of DNS issues.

So now I know. I hope this comment helps others know they are not alone.

[scottsweb]

I am also suffering from this issue on Ubuntu 16.10. With dnsmasq running I occasionally get DNS errors that require dnsmasq to be restarted.

From the help documentation I spotted:

Note that the package "dnsmasq" interferes with Network Manager which can use "dnsmasq-base" to provide DHCP services when sharing an internet connection.

Which I will test and report back if it works.

[javi-dev]

I've been testing the fix on this stackoverflow answer for a couple of days now, and I'm happy to report stable dns and valet-linux running without problems.

TL;DR: Just disable systemd-resolved by running sudo systemctl disable systemd-resolved.service in the console. There seems to be a way to run both, but I haven't bothered because I didn't understand it, it was meant for Arch, and everything was working well enough for my needs.

[scottsweb]

@javi-dev thanks for the info. Have disabled system-resolved too, so far all seems OK. Will report back if anything appears broken or the issue returns.

[cpriego]

@javi-dev @scottsweb I reopened the issue. If anyone can try this solution please report back. Specially with the latests versions of Ubuntu and Fedora.

[scottsweb]

@cpriego Thanks for opening the issue again. So far @javi-dev's solution of sudo systemctl disable systemd-resolved.service is working for me. I had one DNS issue after waking my laptop but I think it was probably unrelated. Valet continues to work well.

If I can help with testing anything then please let me know. Running Ubuntu 16.10.

[javi-dev]

No problems here either on the long term (still on 16.10). I even `valet share`d a WIP to ngrok, and it tunneled correctly. It would be nice if others can confirm it works, or if someone thinks disabling the service is a terrible idea for some reason. I can't install another system on my machine right now, and I don't think a VirtualBox install would prove anything, sorry.

[huepf]

I installed the release version of 17.04 and also see issues with the DNS resolver. External sites are resolved fine but internal names can't be resolved. It doesn't matter if I try just the name or the FQDN. If I check systemd-resolve --status things seem correct. I got the internal DNS server and the DNS domain there correctly.
After I entered the DNS server in resolv.conf nslookup seems to be working but ping doesn't resolve internal names. Even entering the DNS server in resolv.conf should not be necessary if I get the iformation via DHCP!

$ ping zeus
ping: zeus: Name or service not known

$ nslookup zeus
Server:		10.10.0.11
Address:	10.10.0.11#53

Name:	zeus.domain.com
Address: 10.10.0.10

[huepf]

After some digging it turned out to be an issue with DNSSEC. The current version of systemd-resolve seems to expect a working DNSSEC config of your DNS servers. Not signed replies are rejected. Had to add DNSSEC=off to resolved.conf

[EladKeyshawn]

@huepf Hey can you tell me which resolved.conf do you mean?

[EladKeyshawn]

since I opened this file now and it says you should not edit it by hand since its gonna be overwritten
anyways.

[huepf]

It's in /etc/systemd/resolved.conf

[EladKeyshawn]

thanks

[EladKeyshawn]

and then do I need to restart something ?

[huepf]

Restart networking with service systemd-networkd restart or just restart your machine.

[tomicakorac]

Guys, what's the current status of this issue? I was hesitant to upgrade Ubuntu because I didn't want to mess up my Valet configuration, but I did upgrade yesterday, and everything seems to be working just fine. I didn't apply any fix or work-around. Is anyone still experiencing any problems of this kind? My Valet version is 2.0.12.

[cpriego]

@tomicakorac hey, that is great news! Did you upgrade Ubuntu via PackageManager or did you do a clean reinstall? I'm asking this because I haven't changed a thing on valet's dns configuration. If it is working now it means the previous problems were on a misconfiguration on the Ubuntu side.

[tomicakorac]

@cpriego I've done sudo do-release-upgrade -d, if that's what you mean by PackageManager. So not the clean reinstall. Would that mean that the issue might still persist on a new installation of Zesty?

[cpriego]

@tomicakorac i think it might, but I'll do some testing in a VM

[huepf]

I can confirm that the issues are gone since Thursday when an updated version of systemd-resolved was released. Tested it on an updated and fresh installed system. The issue can be closed.

[OndraZizka]

I have these issues now with the official 17.04 release. Even Try Ubuntu mode fails to connect to any server - DNSSEC validation failed.

[OndraZizka]

Ok I read about DNSSEC. What I don't get is why ubuntu servers can't be resolved either. Can someone post some explanation of what's going on, for those directed to here by google?

[pixelrebel]

Just got hit by this. Going from 16.04 to 17.04.

Is this only an issue when upgrading?

[simonpo]

Re: "Is this only an issue when upgrading?" - no, I hit it on a brand new install of 17.04 on a machine that had never run Ubuntu before. After no success with the suggestions in this thread, I downloaded & installed 16.04 instead.

[pixelrebel]

@simonpo Is possible to workaround by disabling DNSSEC. It worked for me. Frustrating though that Canonical released this mess into the public without caring to fix it.

[cpriego]

@simonpo I'll be review this topic once again. As @pixelrebel mentioned, the DNSSEC and systemd-network are a complete mess and every time I fix the issue an update makes breaking changes.

[uldics]

2017.08.17. Found another solution, none from above helped me.
In NetworkManager.conf I added a line: dns=systemd-resolved, and it works now!
No more need to open terminal after each login and type:
echo "nameserver 192.168.0.1" | sudo tee /etc/resolv.conf
Found clue for the line in a solution using unbound on http://www.hecticgeek.com/2017/04/ubuntu-17-04-systemd-dns-issues/.
2017.10.14. Looks like it is not helping afterall. Sometimes PC starts OK, all working, but after a while same problem appears again. Sometimes right after start. Most of the time still need to overwrite the resolv.conf with my routers local address.

[datashaman]

Tried all the above, nothing worked, or if it worked, it resulted in DNS queries that take 15-20 seconds to resolve. This fixed everything, but now I can't use valet-linux. 😢

valet uninstall
composer global remove cpriego/valet-linux
sudo apt purge dnsmasq
sudo systemctl restart network-manager
sudo systemctl restart systemd-resolved

Brand new installation of Ubuntu 17.04. No upgrade.

EDIT: Still issues! Argh! Disabling DNSSEC to see if that works reliably. Perhaps I can re-add valet-linux if that proves successful.

[dakira]

Since the first fix to this issue I had no problems whatsoever. Maybe the contents of the relevant files on my system help:

/etc/NetworkManager/NetworkManager.conf

[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=false

/etc/NetworkManager/conf.d/valet

[main]
dns=dnsmasq

/etc/NetworkManager/dnsmasq.d/valet

address=/.dev/127.0.0.1

/etc/systemd/resolved.conf

[Resolve]
DNS=127.0.1.1
FallbackDNS=127.0.0.1

That's all the DNS relevant configuration I have to have it working on all my machines.

[dakira]

Triage:

1. System booted from 17.04 live USB key.
2. activate universe and multiverse package sources
3. sudo apt install php-cli php-curl php-json php-mbstring php-mcrypt php-sqlite3 php-xml php-zip
4. install composer
5. composer global require cpriego/valet-linux
6. export PATH=/home/ubuntu/.config/composer/vendor/bin:$PATH
7. valet install

Result

1. ping foo.dev resolves correctly to 127.0.0.1
2. valet park'ed a folder and created new/index.php in there with some sample content.
3. opened new.dev in browser, displayed correctly.

The relevant config files all appeared exactly as in my previous comment. Unless there's any new info, this ticket can be closed. @cpriego

[cpriego]

I've had very little free time. It is really helpful when you can test this in fres environment. Thanks @dakira

I'm closing this once again as it is not a Valet issue.

[cpriego]

I have a preview version available in the dnsmasq branch. I've tested it enough to make sure it breaks nothing in Ubuntu, Arch and Fedora.

Could you guys help me by using it? I still need to update the tests.

You can install the version with the following commands. Anyone is welcomed to try it.

composer g require "cpriego/valet-linux:dev-dnsmasq"
valet install

[dakira]

@cpriego I'm going through the changes right now. Any reason you're not using NetworkManager anymore and instead use dnsmasq directly?

Update: Just so that no one else does this work in vain. I'll run the following tests:

• install and test on live 16.04 and 17.04
• install current and try to upgrade

[dakira]

@cpriego This does not work at all for 17.04. Pinging *.dev return a DNS error (i.e. reply from 127.0.53.53).

Can you give me a meta overview of what exactly you're trying to do in this branch? The commits are all a mixture of just cleaning up and completely changing the configuration. Again, testet on a live system.

[cpriego]

@dakira the reason of this branch existence is to fix a problem with NetworkManager and dns resolution when a computer boots off-line.

Relying on NetworkManager in an off-line situation implied DnsMasq was not started, hence no access to any configured TLD (.dev for example). I faced that very same situation when depending on systemd-resolved.

Indeed the branch is a mixture of cleanup and configuration change because I needed to push frequently in order to try the different solutions in different distros (Ubuntu 17.04, Fedora and Arch) making sure nothing broke. So far this proved to be effective in correcting the quirks with Ubuntu 17.04, that's why I asked you guys to test it.

Here is a summary of the changes done:

• Install and enable dnsmasq
• Backup /etc/resolv.conf and place a custom one with write protection (pointing to 127.0.0.1)
• Configure dnsmasq to forward DNS requests to servers obtained from NetworkManager (so the user does not lose name resolution to the rest of the internet).
• Disable and stop systemd-resolved (as this causes conflict with everything).

I tested these changes in virtualboxtaking snapshots of fresh installations. With every modification I simply restored the snapshot and tested in a clean environment, I'm not sure it will work in a live image.

Of course, after every update I ran valet install

[dakira]

It should work in a live image. If it doesn't, something is wrong. Testing with virtualbox VMs is problematic because name resolution extends to the host, AFAIK.

[dakira]

I have a system where I completely disabled systemd-resolved in favor and dnsmasqd. This is all I have to do to achieve that:

sudo systemctl stop systemd-resolved.service
sudo systemctl disable systemd-resolved.service
sudo systemctl stop dnsmasq.service
sudo systemctl disable dnsmasq.service
sudo mv /etc/resolv.conf{,.bak}
sudo ln -s /var/run/NetworkManager/resolv.conf /etc/resolv.conf
echo -e "[main]\ndns=dnsmasq\n" | sudo tee /etc/NetworkManager/conf.d/enable-dnsmasq.conf
sudo systemctl restart NetworkManager.service

[cpriego]

Indeed, your solution works almost flawlessly, except it introduces an issue.

Here are some of the different scenarios I tried and its issues.

Config	Issue
Depending on NetworkManager dns=dnsmasq	Creates issue #86
Depending on /var/run/NetworkManager/resolv.conf	No easy way to prepend 127.0.0.1
Depending on systemd-resolved	Creates issue #86
Adding 127.0.0.1 to /etc/resolvconf/resolv.conf.d/head	Not reliable in all systems
Adding 127.0.0.1 to resolv.conf.head	Not reliable in all systems

In the end what I do is

sudo systemctl stop systemd-resolved.service
sudo systemctl disable systemd-resolved.service
# Install dnsmasq
sudo systemctl enable dnsmasq.service
sudo mv /etc/resolv.conf{,.bak}
echo -e "nameserver 127.0.0.1\n" | sudo tee /etc/resolv.conf
sudo chattr +i /etc/resolv.conf
# Make sure dnsmasq reads config files from /etc/dnsmasq.d/
# Copy custom dnsmasq config to /etc/dnsmasq.d/options
echo -e "[main]\ndns=default\n" | sudo tee /etc/NetworkManager/conf.d/valet.conf
sudo systemctl restart NetworkManager.service
sudo systemctl restart dnsmasq.service

The above commands work across-distro since reading nameservers from /etc/resolv.conf is the standard, then we simply use dnsmasq as our private dns server that resolves whatever TLD (.dev for example) to 127.0.0.1, and everything else is forwarded to the nameservers from NetworkManager (via the resolv-file=/run/NetworkManager/resolv.conf option in dnsmasq)

[dakira]

I'm out of time to test right now, but I'll try to figure out why this doesn't work on vanilla Ubuntu 17.04.

[cpriego]

@dakira I found the reason the version was not working in the live images: the filesystem. In a live image you cannot run the chattr command to lock a file.

[dakira]

Might be, but that's a very hacky way anyway. I think I found a better way. Leave /etc/resolv.conf as it is (symlinked to /run/resolvconf/resolv.conf) and symlink /etc/resolvconf/resolv.conf.d/tail to /run/NetworkManager/resolv.conf.

The resulting /etc/resolv.conf (generates on nm-restart or sudo resolvconf -u) contains nameserver 127.0.0.1 as its first line and the NetworkManager generated resolv.conf after that. At least on my live-system.

Two things I couldn't figure out: On every NetworkManager restart, systemd-resolved also started, even though it was stopped and disabled before. I also couldn't figure out where the 127.0.0.1 in the combined resolv.conf came from.

[cpriego]

@dakira I'm afraid that only works in Ubuntu for is the only one that has an /etc/resolvconf/resolv.conf.d/ directory. Not every distro has a resolvconf file.

Rewriting the resolv.conf file and protecting it from write is the most effective and accepted solution in every document I read.

Preserve_DNS_settings

[dakira]

@cpriego the problem with that are mobile environments. You need some way of dynamically incorporating DNS changes by NM because different routers give you different DNS-Servers or search domains. That breaks if you just write-protect the resolv.conf file. I'm afraid there is no one-size-fits-all-distros solution for this.

[carlihland]

I've been franticly searching to find a solution to dns problems related to vpnc tunneling in Ubuntu Gnome 17.04 and been following this thread. I saw @cpriego 's post and tried on my live image. And it works! So I thought to myself, good, I can finally upgrade to 17.04. Guess what... After installation, I follow the same steps, and it doesn't work again. Why on earth would this work on the live image and not after installation? Does the package updates break this somehow? I know this is a bit off the topic but I getting desperate. I can't seem to find any info anywhere. Thanks a heap for any responses!

[dakira]

@carlihland can you compare your configuration with what I posted above? What version of valet are you using, the current release or the cpriego/valet-linux:dev-dnsmasq branch?

[carlihland]

@dakira Sorry about cluttering this thread with off-topic. I am not using valet but I saw a clear relation between what I am experiencing with my vpnc tunnel and the problems described here concerning dns resolve. It actually looked like it would solve my problems as well. And it kinda did...on the live image. I'm already reverting to 16.04.3 as that release has no issues whatsoever in relation to dns, slow boot and such. Thanks!

[cpriego]

@carlihland indeed, the 17.04 is a mess, solutions work one day but not the next, all this because of updates and the unpredictable situations generated by system-resolved. The live image works because it has no updates applied.

[carlihland]

@cpriego Yup, that was my conclusion as well. 17.04 has some really nice improvements that I'm aching to start using, but DNS not resolving through my much needed tunnel is a deal-breaker for me. Anyway, again I'm sorry for the cluttering. Feel free to tidy up and delete the posts. :)

[pedzed]

Same issue here.

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 17.10
Release:	17.10
Codename:	artful

[dakira]

@pedzed Hey. This specific issue has been solved and closed. Can you describe your issue a bit more. I think opening a new one would be best.