DNS Server Freezing After 1-3 Hours

[jasonlaguidice]

I've been trying out Technitium - it is runnning in an LXC on a Proxmox hypervisor. In this situation I installed it using formerly tteck's but now the Community Scripts at https://community-scripts.github.io/ProxmoxVE/scripts?id=technitiumdns

It is currently setup to be only internally accessible. I added Quad9's IPv4 and IPv6 addresses to the upstream list. I added a couple of hostnames I want to resolve differently internally (effectively split horizon).

After 1-3 hours the server stops responding to DNS requests, the web frontend also becomes unresponsive. Low CPU utilization shows for the container. It remains this way until connecting to the container via SSH to restart Technitium or simply restarting the container entirely.

The logs do not seem to contain useful information - just that a random DNS request fails and a stack trace. Sometimes there are several requests that fail prior to the server becoming unresponsive.

Below is a log where I started the server, cleared the log, and let it run until it froze up some time overnight. In the morning I terminated the container, mounted the filesystem, and pulled the log file. This is the entire log up to when it froze.

Given there is a DNSSEC error related to quad9 I found similar issues #890 and #1002. Those may explain the DNSSEC error but not the full-server lock up.

[2025-03-03 01:03:41 Local] Logging started.
[2025-03-03 01:03:41 Local] DNS Server failed to resolve the request 'aa015h6buqvih86i1.api.met.no. A IN' using forwarders: dns.quad9.net (9.9.9.9), dns.quad9.net (149.112.112.112), dns.quad9.net ([2620:fe::fe]), dns.quad9.net ([2620:fe::9]), 192.168.1.1.
TechnitiumLibrary.Net.Dns.DnsClientResponseDnssecValidationException: Attack detected! DNSSEC validation failed due to missing RRSIG for owner name: aa015h6buqvih86i1.api.met.no/CNAME
   at TechnitiumLibrary.Net.Dns.DnsClient.DnssecValidateSignatureAsync(DnsDatagram response, IReadOnlyList`1 records, IReadOnlyList`1 dnsKeyRecords, IReadOnlyList`1 unsignedZones, DnssecValidateSignatureParameters parameters, Boolean isAuthoritySection, Boolean isAdditionalSection) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 3191
   at TechnitiumLibrary.Net.Dns.DnsClient.DnssecValidateSignatureAsync(DnsDatagram response, IReadOnlyList`1 dnsKeyRecords, IReadOnlyList`1 unsignedZones) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 2962
   at TechnitiumLibrary.Net.Dns.DnsClient.DnssecValidateResponseAsync(DnsDatagram response, IReadOnlyList`1 lastDSRecords, DnsClient dnsClient, IDnsCache cache, UInt16 udpPayloadSize, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 2771
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass95_0.<<InternalDnssecResolveAsync>b__0>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5136
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4708
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4878
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4574
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Func`3 getValidatedResponseAsync, Boolean doNotReorderNameServers, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5027
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Func`3 getValidatedResponseAsync, Boolean doNotReorderNameServers, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4969
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalDnssecResolveAsync(DnsQuestionRecord question, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5110
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass97_0.<<InternalCachedResolveQueryAsync>b__0>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5233
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.ResolveQueryAsync(DnsQuestionRecord question, Func`2 resolveAsync) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4382
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalCachedResolveQueryAsync(DnsQuestionRecord question, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5215
   at DnsServerCore.Dns.DnsServer.DefaultRecursiveResolveAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, IDnsCache dnsCache, Boolean dnssecValidation, Boolean skipDnsAppAuthoritativeRequestHandlers, CancellationToken cancellationToken) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3429
   at DnsServerCore.Dns.DnsServer.RecursiveResolverBackgroundTaskAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, Boolean advancedForwardingClientSubnet, IReadOnlyList`1 conditionalForwarders, Boolean dnssecValidation, Boolean cachePrefetchOperation, Boolean cacheRefreshOperation, Boolean skipDnsAppAuthoritativeRequestHandlers, TaskCompletionSource`1 taskCompletionSource) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3242
[2025-03-03 04:28:44 Local] DNS Server failed to resolve the request '3.a.f.9.8.f.4.6.b.5.a.8.9.3.c.a.1.a.7.0.0.a.3.7.0.2.c.f.7.0.6.2.ip6.arpa. PTR IN' using forwarders: dns.quad9.net (9.9.9.9), dns.quad9.net (149.112.112.112), dns.quad9.net ([2620:fe::fe]), dns.quad9.net ([2620:fe::9]), 192.168.1.1.
DnsServerCore.Dns.DnsServerException: DNS Server received a response for '3.a.f.9.8.f.4.6.b.5.a.8.9.3.c.a.1.a.7.0.0.a.3.7.0.2.c.f.7.0.6.2.ip6.arpa. PTR IN' with RCODE=ServerFailure from: dns.quad9.net ([2620:fe::9])
   at DnsServerCore.Dns.DnsServer.RecursiveResolverBackgroundTaskAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, Boolean advancedForwardingClientSubnet, IReadOnlyList`1 conditionalForwarders, Boolean dnssecValidation, Boolean cachePrefetchOperation, Boolean cacheRefreshOperation, Boolean skipDnsAppAuthoritativeRequestHandlers, TaskCompletionSource`1 taskCompletionSource) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3254
[2025-03-03 06:07:28 Local] DNS Server failed to resolve the request 'api.gfycat.com. AAAA IN' using forwarders: dns.quad9.net (9.9.9.9), dns.quad9.net (149.112.112.112), dns.quad9.net ([2620:fe::fe]), dns.quad9.net ([2620:fe::9]), 192.168.1.1.
TechnitiumLibrary.Net.Dns.DnsClientResponseDnssecValidationException: Attack detected! DNSSEC validation failed as the response was unable to prove non-existence (NX Domain) for owner name: a588eba61f0fe11e98296122db75ecc3-166728206.us-east-1.elb.amazonaws.com
   at TechnitiumLibrary.Net.Dns.DnsClient.DnssecValidateResponseAsync(DnsDatagram response, IReadOnlyList`1 lastDSRecords, DnsClient dnsClient, IDnsCache cache, UInt16 udpPayloadSize, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 2920
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass95_0.<<InternalDnssecResolveAsync>b__0>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5136
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4708
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4878
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4574
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Func`3 getValidatedResponseAsync, Boolean doNotReorderNameServers, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5027
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Func`3 getValidatedResponseAsync, Boolean doNotReorderNameServers, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4969
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalDnssecResolveAsync(DnsQuestionRecord question, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5110
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass97_0.<<InternalCachedResolveQueryAsync>b__0>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5233
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.ResolveQueryAsync(DnsQuestionRecord question, Func`2 resolveAsync) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4382
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalCachedResolveQueryAsync(DnsQuestionRecord question, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5215
   at DnsServerCore.Dns.DnsServer.DefaultRecursiveResolveAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, IDnsCache dnsCache, Boolean dnssecValidation, Boolean skipDnsAppAuthoritativeRequestHandlers, CancellationToken cancellationToken) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3429
   at DnsServerCore.Dns.DnsServer.RecursiveResolverBackgroundTaskAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, Boolean advancedForwardingClientSubnet, IReadOnlyList`1 conditionalForwarders, Boolean dnssecValidation, Boolean cachePrefetchOperation, Boolean cacheRefreshOperation, Boolean skipDnsAppAuthoritativeRequestHandlers, TaskCompletionSource`1 taskCompletionSource) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3242
[2025-03-03 06:07:28 Local] DNS Server failed to resolve the request 'api.gfycat.com. A IN' using forwarders: dns.quad9.net (9.9.9.9), dns.quad9.net (149.112.112.112), dns.quad9.net ([2620:fe::fe]), dns.quad9.net ([2620:fe::9]), 192.168.1.1.
TechnitiumLibrary.Net.Dns.DnsClientResponseDnssecValidationException: Attack detected! DNSSEC validation failed as the response was unable to prove non-existence (NX Domain) for owner name: a588eba61f0fe11e98296122db75ecc3-166728206.us-east-1.elb.amazonaws.com
   at TechnitiumLibrary.Net.Dns.DnsClient.DnssecValidateResponseAsync(DnsDatagram response, IReadOnlyList`1 lastDSRecords, DnsClient dnsClient, IDnsCache cache, UInt16 udpPayloadSize, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 2920
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass95_0.<<InternalDnssecResolveAsync>b__0>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5136
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4708
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4878
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass93_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4574
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Func`3 getValidatedResponseAsync, Boolean doNotReorderNameServers, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5027
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Func`3 getValidatedResponseAsync, Boolean doNotReorderNameServers, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4969
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalDnssecResolveAsync(DnsQuestionRecord question, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5110
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass97_0.<<InternalCachedResolveQueryAsync>b__0>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5233
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.ResolveQueryAsync(DnsQuestionRecord question, Func`2 resolveAsync) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4382
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalCachedResolveQueryAsync(DnsQuestionRecord question, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 5215
   at DnsServerCore.Dns.DnsServer.DefaultRecursiveResolveAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, IDnsCache dnsCache, Boolean dnssecValidation, Boolean skipDnsAppAuthoritativeRequestHandlers, CancellationToken cancellationToken) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3429
   at DnsServerCore.Dns.DnsServer.RecursiveResolverBackgroundTaskAsync(DnsQuestionRecord question, NetworkAddress eDnsClientSubnet, Boolean advancedForwardingClientSubnet, IReadOnlyList`1 conditionalForwarders, Boolean dnssecValidation, Boolean cachePrefetchOperation, Boolean cacheRefreshOperation, Boolean skipDnsAppAuthoritativeRequestHandlers, TaskCompletionSource`1 taskCompletionSource) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 3242

Edit: Following up to add - I restarted the server today and tried again and the logs contain nothing of use as to what the server may be doing when it freezes. Is there a more verbose logging I can enable or some other way to determine why the process may be stuck?

[ShreyasZare]

Thanks for the post. The DNSSEC related error logs are just operational issues so are not related to the issue you see of DNS server freezing. Since the app itself is freezing, it wont be able to log anything so you need to check the issue at the OS level logging.

When the next time the issue occurs, run systemctl status dns to see the status of the dns service from inside the container. Use journalctl --unit dns --follow command too to see if it shows any errors. Do post anything you see with these commands.

Also, let me know how the load on the DNS server is. Is the server getting too many requests? Is the memory/CPU usage very high during the issue? Any such info could be useful to understand the issue better.

[jasonlaguidice]

The service status says it's running happily. Journalctl reports nothing (two boots just for clarity):

-- Boot 4d7d471593b44c119c8dc5d94d28d597 --
Mar 03 09:49:27 technitiumdns systemd[1]: Started dns.service - Technitium DNS Server.
Mar 03 09:49:30 technitiumdns dns-server[98]: Technitium DNS Server was started successfully.
Mar 03 09:49:30 technitiumdns dns-server[98]: Using config folder: /etc/dns
Mar 03 09:49:30 technitiumdns dns-server[98]: Note: Open http://technitiumdns:5380/ in web browser to access web console.
Mar 03 09:49:30 technitiumdns dns-server[98]: Press [CTRL + C] to stop...
-- Boot d42b1f069ee04140a97cc989a7fa5c61 --
Mar 04 09:21:18 technitiumdns systemd[1]: Started dns.service - Technitium DNS Server.
Mar 04 09:21:21 technitiumdns dns-server[98]: Technitium DNS Server was started successfully.
Mar 04 09:21:21 technitiumdns dns-server[98]: Using config folder: /etc/dns
Mar 04 09:21:21 technitiumdns dns-server[98]: Note: Open http://technitiumdns:5380/ in web browser to access web console.
Mar 04 09:21:21 technitiumdns dns-server[98]: Press [CTRL + C] to stop...
lines 64-112/112 (END)

For both of these instances (as well as currently as of 1:53p local time) the server is frozen with nothing reported on the server service log.

The hypervisor is a low power machine running an Intel J5005 and typically sits at ~30% CPU utilization. The container running Technitium reports using 1-3% of the host CPU. It services ~6000 requests an hour which should effectively all be cached as it's a bunch of IOT stuff hitting a domain that should be split horizon'd to a local address... so even if they weren't cached the server should be able to answer directly.

Funnily enough in the screenshot below you can tell exactly when the server froze.

The DNS solution this is replacing is pihole which I run on a container on the exact same hypervisor. The pihole container uses < 0.2% CPU to do the same work. The pihole container is shut down entirely while the technitium container is running.

[ShreyasZare]

Thanks for the details. 6k req/hr is very low to cause any issues. There are no logs that indicate anything so not really sure what could be the reason for this issue.

I would suggest that you try to install the DNS server natively instead of using container and see how it performs. You can export backup zip file from your existing setup and then restore it on the new setup so that the exact config is applied.

[jasonlaguidice]

Just wanted to report back!

I defied the request (a little) and started my own clean LXC for this purpose instead of using the community scripts. I based it on Ubuntu 24.04 since that's what I had a template for. I restored a backup config from the container which was freezing.

Since then it's been working 100% and I haven't had any issues. Whatever was wrong is something prompted by the community scripts (which isn't really surprising seeing as how the MySQL container scripts don't result in a working deployment, either).

Thanks for helping me take a look!

[ShreyasZare]

Thanks for posting your experience here. This should help anyone with similar issue.