Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

systemd-resolved with DNS over TLS : sometimes (rcode=REFUSED) #25635

Closed
psykotox opened this issue Dec 5, 2022 · 3 comments
Closed

systemd-resolved with DNS over TLS : sometimes (rcode=REFUSED) #25635

psykotox opened this issue Dec 5, 2022 · 3 comments
Labels
bug 🐛 Programming errors, that need preferential fixing resolve

Comments

@psykotox
Copy link

psykotox commented Dec 5, 2022

systemd version the issue has been seen with

251 (251.3-1~bpo11+1)

Used distribution

Debian 11 (bullseye)

Linux kernel version used

5.10.0-19-amd64

CPU architectures issue was seen on

None

Component

systemd-resolved

Expected behaviour you didn't see

No response

Unexpected behaviour you saw

No response

Steps to reproduce the problem

When I enable DNSOverTLS in resolved.conf I got temporary failure in name resolution.
My resolved.conf:

[Resolve]
#Use https://1.1.1.1/dns/
DNS=1.1.1.1 1.0.0.1
#FallbackDNS=
Domains=mydomain.com mydomain.io
#DNSSEC=no
DNSOverTLS=yes
#MulticastDNS=no
#LLMNR=no
#DNSStubListener=yes
Cache=yes
#DNSStubListenerExtra=
#ReadEtcHosts=yes
#ResolveUnicastSingleLabel=no

If I disable DNSOverTLS I have no more issue in DNS resolution.

Additional program output to the terminal or log subsystem illustrating the issue

Received dns UDP packet of size 48, ifindex=0, ttl=64, fragsize=0, sender=127.0.0.1, destination=127.0.0.53
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Got DNS stub UDP query packet for id 28486
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Looking up RR for barbara.mydomain.com IN AAAA.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Cache miss for barbara.mydomain.com IN AAAA
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Firing regular transaction 45699 for <barbara.mydomain.com IN AAAA> scope dns on */* (validate=yes).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 45699.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using DNS server 1.1.1.1 for transaction 45699.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending query via TCP since UDP isn't supported or DNS-over-TLS is selected.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 45699.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Announcing packet size 1472 in egress EDNS(0) packet.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing query...
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Received dns UDP packet of size 47, ifindex=0, ttl=64, fragsize=0, sender=127.0.0.1, destination=127.0.0.53
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Got DNS stub UDP query packet for id 40352
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Looking up RR for cardib.mydomain.com IN A.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Cache miss for cardib.mydomain.com IN A
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Firing regular transaction 59591 for <cardib.mydomain.com IN A> scope dns on */* (validate=yes).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 59591.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using DNS server 1.1.1.1 for transaction 59591.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending query via TCP since UDP isn't supported or DNS-over-TLS is selected.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 59591.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Announcing packet size 1472 in egress EDNS(0) packet.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing query...
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Received dns UDP packet of size 47, ifindex=0, ttl=64, fragsize=0, sender=127.0.0.1, destination=127.0.0.53
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Got DNS stub UDP query packet for id 40113
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Looking up RR for cardib.mydomain.com IN AAAA.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Cache miss for cardib.mydomain.com IN AAAA
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Firing regular transaction 28760 for <cardib.mydomain.com IN AAAA> scope dns on */* (validate=yes).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 28760.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using DNS server 1.1.1.1 for transaction 28760.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending query via TCP since UDP isn't supported or DNS-over-TLS is selected.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 28760.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Announcing packet size 1472 in egress EDNS(0) packet.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing query...
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing incoming packet of size 468 on transaction 56620 (rcode=REFUSED).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Server returned REFUSED, switching servers, and retrying.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Retrying transaction 56620, after switching servers.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Switching to system DNS server 1.0.0.1.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/resolve1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=5 reply_cookie=0 sig>
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Cache miss for dualipa.mydomain.com IN AAAA
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Firing regular transaction 56620 for <dualipa.mydomain.com IN AAAA> scope dns on */* (validate=yes).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 56620.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using DNS server 1.0.0.1 for transaction 56620.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending query via TCP since UDP isn't supported or DNS-over-TLS is selected.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 56620.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Announcing packet size 1472 in egress EDNS(0) packet.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Regular transaction 56620 for <dualipa.mydomain.com IN AAAA> on scope dns on */* now complete with <invalid-reply> from none (unsigned; non-confidential).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending response packet with id 63558 on interface 1/AF_INET of size 48.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Freeing transaction 56620.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing incoming packet of size 468 on transaction 20003 (rcode=REFUSED).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Server returned REFUSED, switching servers, and retrying.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Retrying transaction 20003, after switching servers.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Cache miss for zazie.mydomain.com IN A
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Firing regular transaction 20003 for <zazie.mydomain.com IN A> scope dns on */* (validate=yes).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 20003.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using DNS server 1.0.0.1 for transaction 20003.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending query via TCP since UDP isn't supported or DNS-over-TLS is selected.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 20003.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Announcing packet size 1472 in egress EDNS(0) packet.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Regular transaction 20003 for <zazie.mydomain.com IN A> on scope dns on */* now complete with <invalid-reply> from none (unsigned; non-confidential).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending response packet with id 45190 on interface 1/AF_INET of size 46.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Freeing transaction 20003.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing incoming packet of size 468 on transaction 22306 (rcode=REFUSED).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Server returned REFUSED, switching servers, and retrying.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Retrying transaction 22306, after switching servers.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Cache miss for angele.mydomain.com IN AAAA
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Firing regular transaction 22306 for <angele.mydomain.com IN AAAA> scope dns on */* (validate=yes).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 22306.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using DNS server 1.0.0.1 for transaction 22306.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending query via TCP since UDP isn't supported or DNS-over-TLS is selected.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Using feature level TLS+EDNS0 for transaction 22306.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Announcing packet size 1472 in egress EDNS(0) packet.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Regular transaction 22306 for <angele.mydomain.com IN AAAA> on scope dns on */* now complete with <invalid-reply> from none (unsigned; non-confidential).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Sending response packet with id 9034 on interface 1/AF_INET of size 47.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Freeing transaction 22306.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Processing incoming packet of size 468 on transaction 21963 (rcode=REFUSED).
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Server returned REFUSED, switching servers, and retrying.
Dec 05 14:13:56 server.domain.com systemd-resolved[11463]: Retrying transaction 21963, after switching servers.
@psykotox psykotox added the bug 🐛 Programming errors, that need preferential fixing label Dec 5, 2022
@psykotox
Copy link
Author

psykotox commented Dec 6, 2022

The problem seems to appear when systemd-resolved switches from one DNS server to the next. But I don't understand why it has errors rcode=REFUSED with DNSOverTLS enabled on Cloudflare DNS

@psykotox psykotox closed this as completed Dec 6, 2022
@psykotox psykotox reopened this Dec 6, 2022
@psykotox psykotox changed the title systems-resolved with DNS over TLS : sometimes (rcode=REFUSED) systemd-resolved with DNS over TLS : sometimes (rcode=REFUSED) Dec 6, 2022
@poettering
Copy link
Member

I am not sure we can do anything about this. If your DNS servers keep returning REFUSED, then there's really nothing we can do, we have to fail the request.

Hence, I think from our side everything works as expected? I think we can close the issue?

@poettering
Copy link
Member

So with #30513 we support EDE, which will give you more detailed information about server failures if available. Moreover, we actually tweak our state engine based on this.

Let's hence close this. If this is reproducible with upcoming v256, please report back and we can look into this again. Please provide updated logs then, that show us the EDE errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug 🐛 Programming errors, that need preferential fixing resolve
Milestone
No milestone
Development

No branches or pull requests
2 participants
@poettering @psykotox