v0.107.30 Clients set upstream DNS query fail

[NV3S]

Prerequisites

• I have checked the Wiki and Discussions and found no answer

• I have searched other issues and found no duplicates

• I want to report a bug and not ask a question

Operating system type

Linux, Other (please mention the version in the description)

CPU architecture

x86

Installation

Docker

Setup

On a router, DHCP is handled by the router

AdGuard Home version

v0.107.30

Description

What did you do?

I deployed ADG(192.168.100.199) on an x86 Debian Docker host and used Watchtower for automatic updates.(wh)

I have set up a group of clients in ADG, using the unique upstream 192.168.100.200:7874(this server both 53/7874 with internal forward ) to handle DNS.

This morning, it was discovered that the devices in this group of clients were unable to perform DNS query. Upon inspection, it appears that the forwarded domain name resolution has failed.（but can manual using the upstream DNS server,its worked OK)
using “nslookup” it reported "Server failed"

ADG query log without these queries
and
with add a new client to the group ,also the same.
try set upstream to 192.168.100.200:53,same

Then I realized through the group email that adg received the push update of 0.107.30 today, so I tried to roll back to 0.107.29 and the problem disappeared.

Updated again to version 0.107.30 and the issue reappeared.

[NV3S]

udpate:
with 107.30 ,I set the clients upsteam DNS to another ,problem disappeared.
There is something between each other

the 192.168.100.200 is a openwrt

[vhansjee]

Same issue here, reverting to previous build resolves issues.

[axllent]

Wow, this took me about an hour today to debug, but I came to the same exact conclusion and so found this discussion. Things in my network were randomly failing to resolve after the update. My AdGuardHome uses a different internal server (dnsmasq) for upstream queries. I've tried both UDP and TPC, same issue on both. I've also had to downgrade back to v0.107.29 and things are working perfectly fine again.

[nwps]

Same issue here. Using Unbound as an upstream server doesn't work anymore. Once I added 8.8.8.8 as an upstream everything started working again.

[yyysuo]

Same here, downgrade solved the issue.

7874 is the clash dns port, in version v0.108.0-b.34, the upstream test is failed but works well,in version v0.108.0-b.35, no log generated.

[Insight16]

@ameshkov @ainar-g there are problems in the latest release :(

[ainar-g]

@NV3S, thanks for the report. We cannot reproduce this exact bug in our tests: custom upstreams for clients seem to work as usual.

You've mentioned that 192.168.100.200 is running OpenWrt, but what DNS server is it using? The same release introduced a bug in OpenWrt, see #5872, so if you're running AGH there as well, that could cause the failures.

If not, please enable verbose logging and check if there are logs related to the failing queries. You can also send the verbose logs to us at devteam@adguard.com, if you don't want to post it publicly.

[GonzoTG]

I use a Ubiquity UDMSE to resolve all LAN addresses and have the entry setup
[[/lh.xxx.com/]]192.168.7.1

When I try to ping and LAN names, the Adguard server chokes/times out. Like others reverting back to v0.107.29 makes everything start working again

[ainar-g]

We have found a potential bug, and are working on a fix.

[Septrum101]

set lan's upstream will cause this problems include use specifying upstreams for domains.

[chesskuo]

I took a while to identify the cause of the problem.

It is when checking the upstream, ADG is using TCP directly to query DNS.
If the service does not support TCP DNS query, it will be judged as a failure.

You can use this command to test that your dns server support tcp query : dig test. @<dns_server> +tcp

[Mosney]

Please withdraw the v0.107.30 update, this critical bug has caused me to spend some time restoring my local network. At least make an effort to avoid the autoupdate prompt in the WebUI, so that others do not encounter the same issue.

[orosseel]

You can use this command to test that your dns server support tcp query : dig test. @<dns_server> +tcp

What should be the result of this test? I just get a NXDOMAIN response when I do the test with the LAN IP address of my Asus router, within an acceptable time. Yet ADH blocks when I set this IP address as upstream DNS and with the private reverse DNS servers.

The only option at the moment seems to be not setting the router as upstream DNS and disabling reverse DNS resolvers to resolve hostnames.

[chesskuo]

Hello @orosseel ,

NXDOMAIN is also ok, it means your dns server can handle tcp query.
This command is just test your dns server response or not.

In my original case, my dns server is in the remote private network.
I need using port forwarding to get dns query, but I only set udp 53 dst-nat on that time.
And I got a upstream check failed when I click test_upstream button.
When I add the other dst-nat for tcp 53, it works.

I am using PowerDNS as my dns server.

And I also test ASUS router as upstream and reverse dns, but I cannot reproduce your problem, sorry QQ

[orosseel]

Hi @chesskuo,

Thank you for your quick response.

My local DNS server (router) answers normally. Even with v0.107.29, I am not experiencing any problems in that regard. So I am forced to revert to this previous version as this bug causes things to crash immediately after startup.

Meanwhile, I have sent log files by e-mail to devteam@adguard.com.

I wish the team good luck in solving this bug.

[g199209]

I took a while to identify the cause of the problem.

It is when checking the upstream, ADG is using TCP directly to query DNS. If the service does not support TCP DNS query, it will be judged as a failure.

You can use this command to test that your dns server support tcp query : dig test. @<dns_server> +tcp

I meet the same issue, and seems that's the root cause.

The upstream DNS I used is running on localhost port 15354, it only support UDP, do not support TCP. Setting 127.0.0.1:15354 as upstream DNS in latest version will cause DNS resolve failed.

root@OpenWrt:~# dig github.com @127.0.0.1 -p 15354

; <<>> DiG 9.18.10 <<>> github.com @127.0.0.1 -p 15354
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26136
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;github.com.			IN	A

;; ANSWER SECTION:
github.com.		2022	IN	A	140.82.121.4

;; Query time: 10 msec
;; SERVER: 127.0.0.1#15354(127.0.0.1) (UDP)
;; WHEN: Thu Jun 08 22:35:08 CST 2023
;; MSG SIZE  rcvd: 55

root@OpenWrt:~# dig github.com @127.0.0.1 -p 15354 +tcp
;; Connection to 127.0.0.1#15354(127.0.0.1) for github.com failed: connection refused.
;; Connection to 127.0.0.1#15354(127.0.0.1) for github.com failed: connection refused.
;; Connection to 127.0.0.1#15354(127.0.0.1) for github.com failed: connection refused.

A lots of dns tools, like smartDNS, chinadns-ng, dnscrypt-proxy... all only support UDP, do not support TCP. I think AdGuradHome should be able to work with them.

[EugeneOne1]

Hello, @NV3S, and others concerned. We've pushed the newest edge build that should fix this issue. Could you please try it and report if it now resolves over UDP correctly? We're also preparing a proper release if this one is fixed.

[orosseel]

@EugeneOne1 It looks like it has been resolved.

@g199209 I think there was something else wrong besides the problems with DNS requests over TCP.

[MiKing233]

Same issue with v0.107.30 on OpenWrt

[NV3S]

edge build works fine, you catch it 🎉发自我的 iPhone在 2023年6月8日，22:50，Eugene ***@***.***> 写道： Hello, @NV3S, and others concerned. We've pushed the newest edge build that should fix this issue. Could you please try it and report if it now resolves over UDP correctly? We're also preparing a proper release if this one is fixed. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[yyysuo]

fixed in new version，thanks。