Windows: stale catch-all NRPT rule after disconnect kills DNS machine-wide and deadlocks the client

[TobiKr]

Before posting

• I searched existing discussions and issues, including closed ones, and checked the relevant docs.
• I believe this is a product bug rather than a configuration or setup question.
• I can reproduce this issue, or for intermittent issues I've included trigger, frequency, and timing details below.
• I removed or anonymized sensitive data from logs, screenshots, and configuration.

Affected area

Client / Agent

Deployment type

Self-hosted - advanced/custom deployment

Operating system or environment

Windows

NetBird version and upgrade status

• Client: 0.78.1 and 0.79.0 (reproduced on both)
• Management: 0.78.1
• Signal: 0.78.1
• Relay: 0.78.1
• Dashboard: v2.92.0
• Upgrade status: reproduced on current version (client 0.79.0)

Did this work before?

Yes, this worked before

Regression details

• Last known working version: client 0.77.1
• First known broken version: client 0.78.0 — the catch-all NRPT rule was added in [client] Add catch-all NRPT rule when NetBird is the primary DNS resolver #7071,
  merged before the v0.78.0 tag. Confirmed broken on 0.78.1 and 0.79.0.
• Recent changes: the Windows client fleet was upgraded 0.77.1 → 0.78.1 on 2026-09-20/21.
  No server-side, network, firewall or IdP changes. The nameserver group that marks NetBird
  the primary resolver has been unchanged for months and was already in place throughout the
  working period on 0.77.x.

Summary

On Windows, the catch-all NRPT rule added in #7071 is left enforced after the client
disconnects. restoreHostDNS deletes the NRPT registry keys, but the Windows DNS client keeps
serving the rules from its in-memory effective policy table, so every DNS query on the machine
is still routed to the NetBird stub resolver on an interface that no longer exists.

The client then cannot resolve its own management host, so it can never reconnect and never
gets a chance to rewrite the rules. The state is self-sustaining until reboot, because the
keys are volatile and only discarded then.

This takes out DNS for the entire machine, not just NetBird.

Current behavior

netbird up fails, indefinitely:

C:\> netbird up
Error: daemon up failed: login backoff cycle failed: rpc error: code = DeadlineExceeded
desc = create connection: dial context: context deadline exceeded

The message does not mention DNS. The actual failure is only visible in the daemon log:

addrConn.createTransport failed to connect to {Addr: "mgmt.example.com:443"} ...
Err: ... nbnet.NewDialer().DialContext: dial tcp: lookup mgmt.example.com: no such host

CreateConnection uses grpc.WithBlock() without FailOnNonTempDialError or
WithReturnConnectionError (client/grpc/dialer.go:46-62), so every underlying dial failure —
DNS, refused, TLS, firewall — is flattened into the same 30s deadline.

All other name resolution on the machine fails at the same time: browsing, Teams, and MDM
check-in included. The only recovery is a reboot, or a temporary hosts-file entry for the
management host to break the deadlock.

Expected behavior

After a disconnect, the NRPT rules the client installed should stop being enforced, not just
be deleted from the registry. In particular the client must never end up unable to resolve its
own management host, since that makes the failure unrecoverable without operator intervention.

Steps to reproduce

1. Configure a self-hosted deployment with one nameserver group marked Primary, so the
   client computes RouteAll = true and installs the catch-all.
2. On Windows 11, run NetBird >= 0.78 and connect.
3. Confirm the catch-all is live:
   Get-DnsClientNrptPolicy -Effective | Where-Object Namespace -eq '.'
4. Disconnect: netbird down, or sleep/resume, network change, or a service restart.
5. Observe the registry stores are now empty while the effective table still lists .:
   • Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig'
   • Get-ChildItem 'HKLM:\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig'
   • Get-DnsClientNrptPolicy -Effective
6. Resolve-DnsName microsoft.com fails. netbird up fails with the error above and never
   recovers.
7. Reboot clears it, because the keys are volatile.

For intermittent issues

• Trigger: any teardown of the WireGuard interface while the catch-all is installed — netbird down, sleep/resume, docking or Wi-Fi change, service restart, client upgrade.
• Frequency: not every teardown, but frequently. One peer's log contains 709 no such host
  events across recurring multi-hour windows over two days. The same log covers five weeks on
  0.77.0/0.77.1 with only isolated transient blips and no sustained window.
• Timing/timestamps: on one peer, the first routing every namespace through <resolver>: DNS resolution is now exclusive to NetBird appears 7 seconds after the first 0.78.1 session
  starts (2026-09-21 10:40:50 → 10:40:57), and sustained outage windows begin the same
  afternoon. On a second peer, a 35-minute unrecoverable outage across repeated netbird up
  attempts, ended only by reboot.

Environment and topology

• Peer A: Windows 11 Enterprise, client 0.79.0 (also reproduced on 0.78.1), corporate Wi-Fi
• Peer B: N/A — the failure is local to a single peer; no peer-to-peer traffic is involved
• Same LAN or different networks: N/A
• NAT/CGNAT/corporate firewall/mobile network: corporate LAN behind NAT. Not a connectivity
  restriction — the same machine reaches the management host normally as soon as DNS works.
• Other VPN software: None
• Firewall, DNS, or endpoint security software: Windows DNS Client (Dnscache). GPO DNS policy
  store present, so the client logs detected GPO DNS policy configuration, using policy store
  and writes to both stores. AD DNS servers supplied by DHCP.
• Routes, DNS, policies, posture checks, or SSH rules involved: one nameserver group marked
  Primary (hence RouteAll), ~169 match domains including internal AD zones and reverse
  lookup zones. disableServerRoutes=true and blockInbound=true applied via MDM. No posture
  checks or SSH rules involved.
• IdP, reverse proxy, or browser involved: Entra ID (Azure AD) OIDC, ingress-nginx. Neither is
  implicated — the failure occurs before any authentication is attempted.

Self-hosted details, if available

• Deployment method: Helm on Kubernetes
• Management/signal/relay/dashboard versions: 0.78.1 / 0.78.1 / 0.78.1 / v2.92.0
• Reverse proxy: ingress-nginx, TLS via cert-manager
• IdP/provider: Microsoft Entra ID
• STUN/TURN/coturn/relay details: self-hosted coturn on a VM; relay reached at
  rels://<mgmt-host>:443/relay
• Relevant component logs: none — the server side was healthy throughout. Other peers stayed
  connected during the affected windows. The failure is entirely client-side DNS.

Logs, status output, or debug evidence

Key evidence from `dns_windows.txt` on a peer in the broken state — both registry stores empty,
170 rules still enforced, and no `wt0` in `interfaces.txt`:


NRPT rules, local policy store
  no rules
NRPT rules, group policy store
  no rules
NRPT policy table in effect
  .                         NameServers: <netbird-resolver>
  .corp.local               NameServers: <netbird-resolver>
  ... 168 more

The routing table is clean: a single default route over Wi-Fi, no stale VPN interface, so
interface selection is not involved.

Cleanup ran and reported success. removeDNSMatchPolicies calls refreshGroupPolicy()
(RefreshPolicyEx) unconditionally, and restoreHostDNS then flushes the DNS cache. Neither
made the resolver drop the deleted rules — a cache flush clears cached answers, not policy.

Root cause pointers

restoreHostDNS (client/internal/dns/host_windows.go:601) removes the keys and flushes the
cache, but nothing forces the DNS client to reload its policy table. Two comments in the tree
already describe this failure mode:

• host_windows.go:624 — "a rule left behind keeps resolving names over an interface that is
  gone, until reboot discards the volatile key"
• debug_windows.go:113-115 — "it keeps being enforced after its key is gone until the
  resolver reloads its policy"

Before #7071 the leftover rules only covered the configured match domains, so the management
host still resolved and the client always recovered on its own. The catch-all . rule turns
the same leftover state into a total DNS outage plus an unrecoverable client.

Also note mgmt.Resolver.osLookup (client/internal/dns/mgmt/mgmt.go:414) uses
net.DefaultResolver, which is itself subject to NRPT, so it provides no bootstrap path out.

### Related issues or discussions

- Related issue/discussion: #7071 — added the catch-all NRPT rule; this is the change that
  turns a leftover-rule bug into an unrecoverable one.
- #7195 — reworked NRPT deletion to enumerate the registry. Addresses *which* keys are removed,
  not whether removal takes effect in the resolver.
- #7196 — added the Windows DNS section to the debug bundle, which is what made this diagnosable.

### Impact

- Affected users/peers: 1.200 Windows users, the majority within 1–2 days of the 0.78 rollout.
  Two bundles analysed in depth.
- Business or production impact: high. The machine loses DNS entirely, not just the VPN, so
  browsing, Teams and MDM check-in all fail. Because MDM check-in is down, affected machines
  cannot be remediated remotely — every case needs a reboot via the service desk.
- Workaround available: partially. `NB_USE_LEGACY_DNS_RESOLUTION=true` avoids the trigger on
  Windows and restores 0.77 behaviour, but does not repair an already-stuck machine. Clearing
  the primary flag server-side also works, but is not always an option: on macOS the
  primary/catch-all group is the only lever that pulls a `.local` AD domain away from
  mDNSResponder, and `host_darwin.go` has had no functional change in that area across any
  recent release.

### Additional context

### Suggested fixes

1. Force a policy reload in `restoreHostDNS` after deleting the keys, and verify against
   `Get-DnsClientNrptPolicy -Effective` rather than assuming deletion took effect.
2. Keep the management, signal and relay domains out of the catch-all, the way `.local` is
   already exempted. The client should never be able to lose the ability to resolve its own
   control plane.
3. Surface the real dial error instead of a bare `DeadlineExceeded`. The masking in
   `CreateConnection` cost a lot of diagnosis.

[riccardomanfrin]

Thanks for the deep analisys @TobiKr,

I've done a few tests and what decides if Windows DNS client recovers is the presence or not of the GPO group policy store registry key SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig.

That part is a Windows mechanism: when the key is absent, a deletion applies immiediately as the local store is read directly and a change applies at once. Conversely, when the key exists the client's rules go into the group policy store, and a change there (adding or removing a rule) reaches the resolver only when the policy engine next applies DNS client policy; writing or deleting the registry key does not by itself trigger that.

The interesting point is that the client does request a policy refresh on teardown (in RefreshPolicyEx), and on my test machine (not domain-joined) that was enough to drop the rules immediately even with the store present — which is why I could only reproduce the stale state by deleting the keys by hand, without that refresh. On your peers it evidently is not enough, and that is the part still to pin down.

Aside that, #7563, merged on 22 September, removes an empty group policy store before deciding where rules
go, so the local store stays authoritative and a change applies at once. That removes the
dependency on the refresh altogether, rather than making it more reliable. It is not in 0.78.x
or 0.79.0, so it ships with the next release.

In the meantime this addresses the condition directly and might even be pushed via MDM once proved it's solving your situation (requires Netbird not running to take effect):

# Mirrors what the client will do from the next release on: remove the group
# policy DNS policy store only when it is completely empty, so the local store
# becomes authoritative again. A store holding anything — rules, values, other
# subkeys — is left alone, and the parent DNSClient key is never touched.

$gpo = 'HKLM:\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig'
$svc = Get-Service NetBird -ErrorAction SilentlyContinue
$wasRunning = $svc -and $svc.Status -eq 'Running'

if ($wasRunning) { Stop-Service NetBird }

$key = Get-Item $gpo -ErrorAction SilentlyContinue
if ($key -and $key.SubKeyCount -eq 0 -and $key.ValueCount -eq 0) {
    Remove-Item $gpo -Force
    Write-Output 'removed empty GPO DNS policy store'
} else {
    Write-Output 'GPO DNS policy store absent or not empty: nothing to do'
}

if ($wasRunning) { Start-Service NetBird }

Please let me know if this is working for your machines.