Repository navigation
Windows: DNS Client service (Dnscache) high CPU usage (multiple cores) #8300
Unanswered
deadbeef84
asked this question in
Issue Triage
Replies: 1 comment
|
Have you tried to install KB5132061 (windows update, November patches) , I did this and rebooted, and the DNS CPU issue went away. I don't use Netbird but this issue was linked to from a similar CPU issue on tailscale. It seemed to have been caused by either the September/October cumulative patches |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Before posting
Affected area
Client / Agent
Deployment type
Self-hosted - quickstart script
Operating system or environment
Windows
NetBird version and upgrade status
Client: 0.81.3 (and also 0.65.0)
Management: 0.60.5
Signal: 0.60.5
Relay: 0.60.5 (?)
Dashboard: 0.60.5 (?)
Did this work before?
Not sure
Regression details
No response
Summary
On Windows, while NetBird is connected to a large network, the Windows DNS Client service (svchost hosting Dnscache) uses 1–5 CPU cores. The load comes from NetBird's network adapter enumeration, not from DNS lookups.
There's also several thousand DNS requests per second to the management dns name, spamming my network DNS server.
I believe this is something that started happening recently, but I cant say exactly what caused it.
Current behavior
I've tried enabling lazy connections in the client, but that did not solve the issue (I believe server has it disabled). After forcing client to use relay (NB_FORCE_RELAY), the issue went away.
This is from Claudes analysis of the issue:
Summary
Every ICE agent creates its own
stdnet.Net, so the 30-second interface cache instdnetalways starts empty. Each new agent therefore runs a full enumeration. On Windows that is oneGetAdaptersAddressescall fornet.Interfaces()plus one more per adapter forAddrs(). Go callsGetAdaptersAddresseswithoutGAA_FLAG_SKIP_DNS_SERVER/GAA_FLAG_SKIP_DNS_INFO, so each call also asks the DNS Client service for every adapter's DNS settings. With a few hundred peers creating agents, the DNS Client service stays busy all the time.Expected behavior
The ICE agents should share one interface cache, so that the 30 s
updateIntervallimits enumeration across all agents instead of per agent. Adapter enumeration on Windows shouldn't load the DNS Client service in proportion to peer count or signaling traffic.Steps to reproduce
DnscacheCPU:Get-CimInstance Win32_PerfFormattedData_PerfProc_Process | ? IDProcess -eq (Get-CimInstance Win32_Service -Filter "Name='Dnscache'").ProcessIdnetbird down, and againstNB_FORCE_RELAY=true.Environment and topology
wt0)wt0, Wi-Fi, Bluetooth PAN, WSL Hyper-V vEthernet)Self-hosted details, if available
quickstart / docker compose
Logs, status output, or debug evidence
Related issues or discussions
Closest issue I've found is #4549, but I'm not seeing the errors, only high cpu usage. I don't think they are the same.
Impact
Set
NB_FORCE_RELAY=truein the NetBird service environment (HKLM\SYSTEM\CurrentControlSet\Services\Netbird,Environmentmulti-string value) and restart the service.Additional context
More from Claude troubleshooting:
Observed
DNS Client service CPU, sampled once per second (100% = one core, 20 logical cores):
netbird down)NB_LAZY_CONN=trueNB_FORCE_RELAY=trueDNS Client spikes line up with NetBird's own CPU bursts. Forcing relay mode, which stops ICE, removes the load completely. Lazy connections had no effect, because the remote peers keep connecting.
Isolating the cause
With NetBird stopped, I loaded the DNS Client service with two kinds of calls:
NetworkInterface.GetAllNetworkInterfaces(), which wrapsGetAdaptersAddresses)Dns.GetHostAddressesof a cached name)Adapter enumeration costs roughly 1,000× more DNS Client CPU per call than a cached lookup. The cost of DNS lookups during ICE (STUN/TURN URLs) is small in comparison.
Code path (v0.81.3)
stdnet.Net:client/internal/peer/ice/agent.go#L42callsnewStdNet(...), which callsstdnet.NewNet(...)(client/internal/peer/ice/stdnet.go#L11-L12)updateIntervalnever applies to a new agent:client/internal/stdnet/stdnet.go#L180-L190GetAdaptersAddressescalls:client/internal/stdnet/discover_pion.go#L12-L36:net.Interfaces(), thenoif.Addrs()for each interfacenet/interface_windows.go, bothinterfaceTableandinterfaceAddrTablecalladapterAddresses(). That function usesGAA_FLAG_INCLUDE_PREFIX | GAA_FLAG_INCLUDE_GATEWAYS, which does not skip DNS server or DNS suffix information.client/internal/peer/worker_ice.go#L223(reCreateAgent), on every remote offer with a new session ID. Remote peers on 0.60.x send one every 30–45 s for every peer they aren't connected to over P2P.client/internal/peer/guard/ice_monitor.go#L81, every 5 minutes (minor)For scale: in a 13-minute window this client received about 1,400 offers. 23 peers on 0.39.1 sent 791, 26 peers on 0.60.5 sent 547, and about 70 peers on 0.80.0 sent about 70. Earlier, while this host still ran 0.65.0, the log showed about 14,800 sent offers per hour and about 13,000 received answers per hour.
Suggested fix
stdnet.Net, or at least one shared interface cache, per engine, and pass it to every ICE agent and the ICE monitor. Invalidate it on network change events.net.Interfaces()+Addrs()inpionDiscoverwith a singlegolang.org/x/sys/windows.GetAdaptersAddressescall that passesGAA_FLAG_SKIP_DNS_SERVER | GAA_FLAG_SKIP_DNS_INFO | GAA_FLAG_SKIP_MULTICAST | GAA_FLAG_SKIP_ANYCAST. That builds the interface and address list in one cheap call.All reactions