resolved’s 127.0.0.53 breaks ‘dig +trace’ by refusing queries on root zone (.) with RD bit #5897

andersk · 2017-05-05T20:52:23Z

Submission type

Bug report
Request for enhancement (RFE)

systemd version the issue has been seen with

232-21ubuntu3

Used distribution

Ubuntu 17.10

In case of bug report: Expected behaviour you didn't see

The dig +trace HOSTNAME command begins by performing a query on the configured nameserver for the root zone NS records with the RD (recursion disabled) bit. This should return a list of nameservers:

$ dig +norecurse . NS @8.8.8.8

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +norecurse . NS @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65282
;; flags: qr ra ad; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;.				IN	NS

;; ANSWER SECTION:
.			15438	IN	NS	a.root-servers.net.
.			15438	IN	NS	b.root-servers.net.
.			15438	IN	NS	c.root-servers.net.
.			15438	IN	NS	d.root-servers.net.
.			15438	IN	NS	e.root-servers.net.
.			15438	IN	NS	f.root-servers.net.
.			15438	IN	NS	g.root-servers.net.
.			15438	IN	NS	h.root-servers.net.
.			15438	IN	NS	i.root-servers.net.
.			15438	IN	NS	j.root-servers.net.
.			15438	IN	NS	k.root-servers.net.
.			15438	IN	NS	l.root-servers.net.
.			15438	IN	NS	m.root-servers.net.

;; Query time: 20 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri May 05 16:48:17 EDT 2017
;; MSG SIZE  rcvd: 239

In case of bug report: Unexpected behaviour you saw

When the configured nameserver is systemd-resolved’s 127.0.0.53, systemd-resolved refuses this query.

$ dig +norecurse . NS

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +norecurse . NS @127.0.0.53
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 37118
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;.				IN	NS

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Fri May 05 16:48:56 EDT 2017
;; MSG SIZE  rcvd: 28

Therefore, dig +trace does not work.

$ dig +trace github.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +trace github.com
;; global options: +cmd
;; Received 28 bytes from 127.0.0.53#53(127.0.0.53) in 0 ms

In case of bug report: Steps to reproduce the problem

Run dig +norecurse . NS or dig +trace github.com.

The text was updated successfully, but these errors were encountered:

poettering · 2017-05-09T17:21:28Z

This is really by design. The DNS stub in resolved is not supposed to be a full DNS server. All it shall be able to do is reply to simply local lookups, i.e. RD lookups done by nss-dns or similar local implementations.

resolved is not a DNS server, and it's not going to become one. Sorry!

poettering · 2017-05-09T17:23:41Z

Will close this as duplicate of #4621, as both are minifestations of the same cause: resolved not being a DNS server, but a only a simply DNS lookup stub...

(And I figure we should log about this fact whenever we receieve such a query, so that people are not surprised...)

lawipac · 2018-05-13T16:21:07Z

I need dig +trace print, but it does not work with resolved's 127.0.0.53.
I tried dnstracer, and it does not work either.

The only way I can achieve dig+trace is to kill 127.0.0.53 in /etc/resolv.conf, and replace it with real dns IP address. In that case, I cannot see the benefits of resolved’s 127.0.0.53.

Could you please advise why we need this 127.0.0.53 in the first place?

MilanKerslager · 2018-06-25T18:47:34Z

@lawipac The 127.0.0.53 is there mainly because Glibc has broken resolver (AFAIK).

TheRealBecks · 2018-06-28T11:57:48Z

It's a real pain in daily business. Linux servers and clients are only showing "127.0.0.53" as DNS server. Therefore it's not possible to debug a DNS problem. I have tested it in (K)Ubuntu 17.10 and Ubuntu server 18.04.

I can understand that resolved is not a DNS server, but that answer doesn't solve the problem at all. Is there any way to solve this behaviour so that newer releases of resolved will resolve the correct IP addresses of the DNS servers?

ZLightning · 2018-07-10T16:17:57Z

This seems like a simple fix. When resolved is queried for the root NS with recursion disabled, just make the query to the real resolver without recursion and return the answer. That doesn't require a massive change in functionality, it just requires treating one query special so that backwards compatibility with popular diagnostic tools actually works as expected.

RE: #5897 (comment)

@poettering, if resolved can't fix the issue that breaks diagnostic tools, then maybe we need to remove the functionality entirely. This simple problem complicates maintenance, security, and stability issues. Breaking such important functionality without a replacement and without real gains is truly unacceptable to system admins.

floppym · 2018-07-10T22:52:53Z

It seems that unbound shows the same "REFUSED" behavior with +norecurse/+trace. So this doesn't really seem to behave consistently among real DNS daemons.

MarkYSA · 2018-10-24T08:17:39Z

In my prod env I shall to disable systemd resolve and replace it with unbound or resolvconf because our staff need dig trace function...

DoktorJ · 2019-03-06T14:31:57Z

Wow, it's "by design" to break existing diagnostic tools and force them to manually change software components (or worse, try to direct others to remotely or over the phone) to be able to run those diagnostics?

This hurts my head so badly. If resolved is incapable of handling the request, it would make sense to me at least for it to redirect, forward, or otherwise mitigate it so that the user gets an appropriate response, not just "derp I don't understand this query" with no useful information on how to get around it.

TheRealBecks · 2019-03-08T11:34:52Z

@lawipac The 127.0.0.53 is there mainly because Glibc has broken resolver (AFAIK).

What does that mean?

toreanderson · 2019-05-13T19:05:54Z

FYI, I submitted a bug report on ISC BIND (which contains the dig tool) about this issue here:

https://gitlab.isc.org/isc-projects/bind9/issues/1028

toreanderson · 2019-05-22T05:40:33Z

A fix for this issue was just merged to ISC BIND Git Master:

https://gitlab.isc.org/isc-projects/bind9/commit/8ddc54e20054e91e6e2dd32a4510a5eecc777548

toreanderson · 2024-07-18T14:22:37Z

Wondering why it doesn't work, guess what, another piece of software that was broken by this complete utter moron.

It didn't work due to a bug in the ISC dig tool. That bug was fixed upstream more than five years ago, cf. https://gitlab.isc.org/isc-projects/bind9/issues/1028.

systemd-resolved has behaved correctly all along.

DoktorJ · 2024-07-18T15:23:22Z

Imagine being so small-minded and petty as to use GitHub as a platform for airing your personal grievances over a problem that's had a solution for five years already 😳

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

resolved’s 127.0.0.53 breaks ‘dig +trace’ by refusing queries on root zone (.) with RD bit #5897

resolved’s 127.0.0.53 breaks ‘dig +trace’ by refusing queries on root zone (.) with RD bit #5897

andersk commented May 5, 2017

poettering commented May 9, 2017

poettering commented May 9, 2017

lawipac commented May 13, 2018 •

edited

Loading

MilanKerslager commented Jun 25, 2018

TheRealBecks commented Jun 28, 2018

ZLightning commented Jul 10, 2018 •

edited

Loading

floppym commented Jul 10, 2018

MarkYSA commented Oct 24, 2018 •

edited

Loading

DoktorJ commented Mar 6, 2019

TheRealBecks commented Mar 8, 2019

toreanderson commented May 13, 2019 •

edited

Loading

toreanderson commented May 22, 2019

This comment was marked as abuse.

This comment has been minimized.

toreanderson commented Jul 18, 2024

This comment was marked as abuse.

DoktorJ commented Jul 18, 2024

resolved’s 127.0.0.53 breaks ‘dig +trace’ by refusing queries on root zone (.) with RD bit #5897

resolved’s 127.0.0.53 breaks ‘dig +trace’ by refusing queries on root zone (.) with RD bit #5897

Comments

andersk commented May 5, 2017

Submission type

systemd version the issue has been seen with

Used distribution

In case of bug report: Expected behaviour you didn't see

In case of bug report: Unexpected behaviour you saw

In case of bug report: Steps to reproduce the problem

poettering commented May 9, 2017

poettering commented May 9, 2017

lawipac commented May 13, 2018 • edited Loading

MilanKerslager commented Jun 25, 2018

TheRealBecks commented Jun 28, 2018

ZLightning commented Jul 10, 2018 • edited Loading

floppym commented Jul 10, 2018

MarkYSA commented Oct 24, 2018 • edited Loading

DoktorJ commented Mar 6, 2019

TheRealBecks commented Mar 8, 2019

toreanderson commented May 13, 2019 • edited Loading

toreanderson commented May 22, 2019

This comment was marked as abuse.

This comment has been minimized.

toreanderson commented Jul 18, 2024

This comment was marked as abuse.

DoktorJ commented Jul 18, 2024

lawipac commented May 13, 2018 •

edited

Loading

ZLightning commented Jul 10, 2018 •

edited

Loading

MarkYSA commented Oct 24, 2018 •

edited

Loading

toreanderson commented May 13, 2019 •

edited

Loading