FQDN-based egress control for Kubernetes, on any CNI.
Write egress rules against hostnames instead of hand-curated CIDR lists. The controller resolves
them to IPs, tracks DNS TTLs, and reconciles a standard networking.k8s.io/v1 NetworkPolicy —
so enforcement stays with whatever CNI you already run: Calico, AWS VPC CNI, Azure CNI, Flannel,
or anything else that honors NetworkPolicy.
apiVersion: netsec.kunal.dev/v1alpha1
kind: FQDNNetworkPolicy
metadata:
name: allow-payment-apis
namespace: payments
spec:
mode: Enforce
podSelector:
podSelector:
matchLabels:
app: checkout
egress:
- match: api.stripe.com
ports:
- port: 443
protocol: TCP
- match: api.github.com
ports:
- port: 443
protocol: TCPTable of contents
- Why this exists
- How it compares
- Architecture
- kubectl plugin
- Installation
- Writing policies
- Resolver strategy
- Resolver disagreement
- ASN/org enrichment (opt-in)
- Discovering egress domains:
FQDNEgressObservation - Detecting egress drift
- Spec reference
- Status and observability
- Admission webhook
- High availability
- Production deployment checklist
- Project layout
- Development
- Known limitations
Standard Kubernetes NetworkPolicy accepts only CIDR ipBlock rules. External APIs — Stripe,
GitHub, S3, auth providers — rotate IPs continuously with short DNS TTLs, run behind CDNs with
anycast routing, and return different IPs to different callers. Maintaining hand-curated CIDR
lists is error-prone and breaks silently. This controller keeps those lists current automatically.
| Feature | fqdn-network-policy | Cilium FQDNNetworkPolicy | Calico FQDN policy | Antrea FQDN | Istio Egress |
|---|---|---|---|---|---|
| CNI requirement | None — any CNI | Cilium only | Calico only | Antrea only | None |
| Sidecar required | No | No | No | No | Yes (Envoy) |
| DNS resolution method | Active + 4-resolver union + eBPF snoop | eBPF kernel intercept | DNS poll | DNS proxy | SNI inspection |
Produces standard NetworkPolicy |
Yes | No — Cilium CRD | No — Calico CRD | No — Antrea CRD | No — VirtualService |
| CDN / anycast accuracy | ~85 % (4 resolvers) | Near-exact | ~70 % | Moderate | SNI-based |
| Wildcard rules | Prefix expansion / snoop | Yes | Yes | Yes | Yes |
| Cluster-wide policy | Yes (ClusterFQDNNetworkPolicy) |
Yes | Yes | Yes | Yes |
kubectl preview plugin |
Yes — kubectl fqdn-policy preview |
No | No | No | No |
| DNS rebinding protection | Yes — blockPrivateIPs default |
Partial | No | No | N/A |
| CNAME chain visibility | Yes — surfaced in status | No | No | No | No |
| Multi-upstream resolver | Yes — unions Cloudflare, Google, Quad9, OpenDNS | No | No | No | No |
| Infrastructure additions | None | Replace CNI | Replace CNI | Replace CNI | Service mesh |
Tip
Choose Cilium if you need exact per-pod DNS accuracy at scale and can replace your CNI. Choose this project if you need portable egress control on the CNI you already run, with zero infrastructure additions.
When to use this instead of Cilium or Istio:
- You run Calico, AWS VPC CNI, Azure CNI, or Flannel and cannot or do not want to replace your CNI.
- You need lightweight egress control with zero infrastructure additions (no sidecars, no service mesh, no CNI replacement).
- Your egress targets are stable internal APIs, corporate auth providers, or fixed SaaS endpoints.
When to use Cilium instead: if you need wildcard accuracy for CDN-backed services (Cloudflare, Fastly, AWS CloudFront) at production scale, Cilium's eBPF DNS snooping is the correct tool. This controller includes a multi-upstream resolver that covers ~85% of CDN divergence cases but is not equivalent to in-kernel packet interception.
flowchart TD
YAML["FQDNNetworkPolicy /<br/>ClusterFQDNNetworkPolicy YAML"] -->|kubectl apply| Webhook
subgraph CP[Control plane]
Webhook["Admission webhook<br/>syntax, wildcards, ports"]
Controller["Controller<br/>reconcile loop"]
Webhook --> Controller
end
subgraph RES[Resolution]
Resolver["Resolver<br/>Active / Multi / CoreDNS / Snoop"]
Filter["Security filter<br/>private IPs, consensus, divergence"]
Builder["NetworkPolicy builder"]
Resolver --> Filter --> Builder
end
Controller --> Resolver
Builder --> NP["networking.k8s.io/v1<br/>NetworkPolicy"]
subgraph DNS[DNS path]
CoreDNS["CoreDNS"]
Snoop["SnoopResolver proxy"]
Upstream["Upstream resolvers"]
CoreDNS -->|forward| Snoop --> Upstream
end
Resolver -->|active lookups| Upstream
Snoop -.->|observed IPs + hostnames| Resolver
subgraph DP[Data plane]
CNI["CNI<br/>Calico, AWS VPC CNI, Azure CNI, ..."]
Pod["Workload pod"]
Pod -->|DNS query| CoreDNS
end
NP --> CNI
CNI ==>|egress enforced| Pod
classDef hot fill:#1f6feb,color:#fff,stroke:#1f6feb;
class Controller,Builder hot
Note
Fail-closed: if no hosts have resolved yet (first reconcile, cold start, all lookups failed), the controller emits a deny-all-egress policy rather than a permissive no-op. DNS egress (UDP/TCP port 53) is always permitted in the generated policy so resolution can continue.
Transient DNS safety: a resolution failure retains the last known-good IPs for that host
rather than revoking access. The Degraded status condition is set while this fallback is active.
kubectl fqdn-policy lets you inspect what a policy will do before applying it.
make kubectl-plugin
cp bin/kubectl-fqdn_policy /usr/local/bin/kubectl fqdn-policy preview policy.yamlPolicy: allow-payment-apis
Namespace: payments
api.stripe.com
A: 3.18.12.10, 3.18.12.11
AAAA: 2600:1f18:2148:bc01::a
TTL: 38s
CNAME: (none)
Resolver divergence: 4 IPs across 4 resolvers
Generated NetworkPolicy would allow:
3.18.12.10/32:443/TCP
3.18.12.11/32:443/TCP
2600:1f18:2148:bc01::a/128:443/TCP
Warnings:
[WARN] api.stripe.com: resolver divergence (4 IPs appeared in fewer than all resolvers)
Flags:
--coredns-address <host:port>— query cluster CoreDNS instead of public resolvers--no-multi-resolver— use the system resolver only
kubectl fqdn-policy diff payments/allow-payment-apis
# or
kubectl fqdn-policy diff allow-payment-apis -n paymentsPolicy: allow-payment-apis (namespace: payments)
api.stripe.com
+ 54.160.0.1/32 (new)
- 3.18.12.11/32 (stale)
Warnings shown for: private / loopback / link-local IPs, resolver divergence, CNAME chain depth > 3.
kubectl fqdn-policy suggest payments/checkoutReads the named FQDNNetworkPolicy's status.observedUnpoliciedDomains (see
Detecting egress drift) and prints a suggested FQDNNetworkPolicy
manifest — one rule per drifted hostname, starting in Audit mode — for you to review, edit, and
kubectl apply -f - yourself. Nothing is written to the cluster by this command.
Synthesizes a production-ready FQDNNetworkPolicy from observed DNS traffic (FQDNEgressObservation or an offline observation file). Filters cluster-internal DNS names (.cluster.local, arpa, localhost), clusters high-cardinality subdomains into wildcards (*.stripe.com), classifies by provider/category, and applies safe security defaults in Audit mode:
# Discover from live cluster observation
kubectl fqdn-policy profile cluster-egress-observation -n payments --app-label app=checkout
# Run offline / air-gapped from an exported observation file
kubectl fqdn-policy profile --from-file observation.yaml --wildcard-threshold 3 --preset strictFlags:
--from-observation <name>— Name of theFQDNEgressObservationresource in cluster--from-file <path>— Offline YAML/JSON observation file (works in air-gapped / CI environments)--wildcard-threshold <int>— Subdomains required under the same domain to trigger wildcard rollup (default:3,0to disable)--preset <production|strict|relaxed>— Security preset (default:productionwithblockPrivateIPs: true)--app-label <key=val>— SetpodSelector.matchLabels(e.g.app=checkout)--mode <Audit|Enforce>— Policy mode (default:Audit)-o, --output <file>— Write policy YAML to file instead of stdout
Performs pre-flight blast radius simulation of a proposed policy against real observed cluster DNS queries (from an active observation or an offline baseline file). Identifies exactly which domains will be permitted and which domains will be BLOCKED, preventing production outages before switching from Audit to Enforce:
# Simulate policy against recorded baseline
kubectl fqdn-policy simulate policy.yaml --baseline observation.yaml
# CI/CD Gate: Fail build if any active domain would be blocked
kubectl fqdn-policy simulate payments/checkout-policy --fail-on-blocked
# Machine-readable JSON output for automated GitOps pipelines
kubectl fqdn-policy simulate policy.yaml --baseline observation.yaml --jsonWhat the simulation checks:
- Blast Radius Analysis: Exact list of actively queried hostnames that are missing from the policy and would be blocked.
- Coverage Breakdown: Total domains, permitted domains, dropped domains, and ignored internal Kubernetes services.
- Security & Sanity Audits: Flags empty egress rules, overly broad wildcards (
*,*.com), and disabled private IP blocking.
helm install fqdn-network-policy charts/fqdn-network-policy/ \
--namespace fqdn-network-policy-system \
--create-namespace \
--set replicaCount=2 \
--set leaderElection.enabled=true \
--set multiResolver.enabled=trueVerify the controller is running:
kubectl get pods -n fqdn-network-policy-system
kubectl get fqdnnetworkpolicies --all-namespaces# Install CRDs
kubectl apply -f config/crd/bases/
# Install RBAC and controller
kubectl apply -f config/rbac/role.yaml
kubectl apply -f config/manager/deployment.yamlmake build # compile the manager binary to ./bin/manager
make run # run against your current kubeconfig context
./demo.sh # full KinD demo with Calico enforcement proofPrerequisites: kind, kubectl, docker on PATH.
Applies to pods in a single namespace.
apiVersion: netsec.kunal.dev/v1alpha1
kind: FQDNNetworkPolicy
metadata:
name: allow-external-apis
namespace: payments
spec:
# mode: Enforce writes a NetworkPolicy. mode: Audit logs only — safe for validation.
mode: Enforce
podSelector:
podSelector:
matchLabels:
app: checkout
egress:
- match: api.stripe.com
ports:
- port: 443
protocol: TCP
- match: api.github.com
ports:
- port: 443
protocol: TCP
# Wildcards require --enable-snoop-resolver=true (see Resolver Strategy below)
- match: "*.s3.amazonaws.com"
ports:
- port: 443
protocol: TCP
# Optional: override DNS TTL-based scheduling. Min 5s, max 300s.
# resolutionTTLOverride: 30
# Optional: resolve against cluster CoreDNS instead of the node resolver.
# coreDNSAddress: "10.96.0.10:53"Enforces baseline egress rules across all matched namespaces. Platform teams use this for corporate telemetry, auth providers, and shared infrastructure without duplicating CRs per namespace.
apiVersion: netsec.kunal.dev/v1alpha1
kind: ClusterFQDNNetworkPolicy
metadata:
name: allow-corporate-telemetry
spec:
mode: Enforce
# Apply to all namespaces labelled env=production
namespaceSelector:
matchLabels:
env: production
podSelector:
podSelector: {} # all pods in matched namespaces
egress:
- match: otel-collector.corp.example.com
ports:
- port: 4317
protocol: TCP
- match: auth.corp.example.com
ports:
- port: 443
protocol: TCPThe controller creates one NetworkPolicy per matched namespace (named cfqdnnp-<policy-name>)
and garbage-collects it if the namespace no longer matches the selector.
Configure the resolver that best matches your infrastructure:
| Resolver | Flag | CDN accuracy | Wildcard | Privilege required |
|---|---|---|---|---|
| SnoopResolver | --enable-snoop-resolver |
Exact — sees actual pod responses | Yes (dynamic) | CoreDNS forward config |
| MultiResolver | --enable-multi-resolver (default) |
~85% — unions 4 public resolvers | Yes (prefix expansion) | None |
| CoreDNS direct | --coredns-address |
Better for internal split-horizon | No | None |
| ActiveResolver | (fallback) | Same as node resolver | No | None |
The SnoopResolver runs a DNS forwarding proxy that CoreDNS routes queries through. It records exactly which IPs pods received in its response stream.
-
Start the controller with snoop enabled:
--enable-snoop-resolver=true --snoop-listen-address=0.0.0.0:5353 --snoop-upstream=<a real upstream, e.g. 8.8.8.8:53> # never point this back at CoreDNS
-
Patch your CoreDNS ConfigMap to forward through the snoop proxy.
forwardonly accepts an IP address (or/etc/resolv.conf), not a Service DNS name — use the snoop Service's ClusterIP:.:53 { forward . <snoop-service-cluster-ip>:5353 cache 30 reload } -
Wildcard rules (e.g.
*.s3.amazonaws.com) are now accepted by the admission webhook and dynamically expanded as the proxy observes new subdomains from pod traffic. -
The snoop resolver is also what backs
FQDNEgressObservationbelow — but note its cluster-wide (not per-pod) limitation before relying on it for discovery.
Queries Cloudflare (1.1.1.1), Google (8.8.8.8), Quad9 (9.9.9.9), and OpenDNS (208.67.222.222) concurrently. Returns the union of all A and AAAA records. For CDN-backed services that use anycast routing, different resolvers return different PoP IPs — unioning them covers ~85% of divergence cases with no additional infrastructure.
Enable in Helm:
multiResolver:
enabled: true # default
coreDNS:
address: "10.96.0.10:53" # also queries cluster CoreDNS when setMultiResolver unions answers from several upstreams, which is exactly the right default for
CDN anycast — but a large IP set can also mean something worth a second look (split-horizon
leak, resolver poisoning, a geo-DNS oddity), and unioning silently threw away which resolver said
what. When resolvers disagree on a hostname, that's now visible:
kubectl get fqdnnetworkpolicy allow-payment-apis -n payments -o yamlstatus:
resolvedHosts:
- hostname: api.stripe.com
security:
resolverDivergence: 2
resolverResults: # only present when resolverDivergence > 0
1.1.1.1:53: ["3.1.4.1", "3.1.4.2"]
8.8.8.8:53: ["3.1.4.1"]
conditions:
- type: ResolverDivergence
status: "True"
reason: ResolverDisagreement
message: "api.stripe.com: 2 resolvers returned 2 IPs total, 1 IPs appeared in only 1 resolver"fqdnnp_resolver_divergence_total{hostname,policy} tracks the same signal in Prometheus.
Opt-in follow-ons, both off by default in spec.security:
blockOnDivergence— freeze the currently-appliedNetworkPolicyfor a reconcile cycle instead of admitting a divergent IP set. Status still updates (so you can see why), but noNetworkPolicywrite happens until the next clean resolution.minResolverAgreement— consensus mode. An IP only makes it into the allow-list once at least this many upstream resolvers agree on it. Requires per-resolver result data (MultiResolver or SnoopResolver); a no-op with a plainActiveResolver/CoreDNSResolver.
Neither changes the default union behavior unless you turn them on.
Annotate resolved IPs with autonomous-system and organization data, so an operator can tell at a
glance whether api.stripe.com resolving to an unfamiliar IP is a new CDN edge (expected) or an
unrelated hosting provider (suspicious). Disabled by default — no external calls unless configured.
--asn-enricher=ipinfo # or FQDNNP_ASN_ENRICHER=ipinfo
--ipinfo-token=<token> # optional, or FQDNNP_IPINFO_TOKEN; raises ipinfo.io rate limitsWhen enabled, each resolved IP is looked up against ipinfo.io asynchronously —
enrichment never blocks policy reconciliation. Results are cached for 6 hours per IP (also the
enrichment rate limit) and populate status.resolvedHosts[].ipEnrichments:
kubectl get fqdnnetworkpolicy allow-payment-apis -n payments \
-o jsonpath='{.status.resolvedHosts[0].ipEnrichments}' | jq{
"54.160.0.1": {
"asn": "AS16509",
"org": "Amazon.com, Inc.",
"country": "US",
"enrichedAt": "2026-09-07T08:39:52Z"
}
}If an IP's ASN/org changes between enrichment cycles (e.g. a hostname moves to a different
hosting provider), the controller logs a warning and increments fqdnnp_asn_change_total.
Blocking policy application on an ASN change, and allowlisting by expected ASN, are both
follow-on features — this only surfaces the data.
Writing an FQDNNetworkPolicy from scratch requires knowing what hostnames your application
resolves. FQDNEgressObservation surfaces that passively from real DNS traffic, via the
SnoopResolver — no code changes, no guessing, no watching for connection
failures after applying a policy.
apiVersion: netsec.kunal.dev/v1alpha1
kind: FQDNEgressObservation
metadata:
name: checkout-observation
namespace: payments
spec:
podSelector:
podSelector:
matchLabels:
app: checkout
observationWindow: 24h # how long to accumulate before status.observationComplete is setkubectl get fqdnegressobservation checkout-observation -n payments \
-o jsonpath='{.status.observedDomains}' | jq[
{ "hostname": "api.stripe.com", "firstSeen": "2026-09-07T09:00:00Z", "lastSeen": "2026-09-07T14:22:00Z" },
{ "hostname": "api.github.com", "firstSeen": "2026-09-07T09:01:00Z", "lastSeen": "2026-09-07T14:22:00Z" }
]Warning
This is cluster-wide, not per-pod. spec.podSelector is accepted for forward compatibility
but has no filtering effect today: CoreDNS's forward plugin re-originates every forwarded
query as its own client, so the snoop proxy can never see which pod actually asked — only that
a query happened, somewhere in the cluster. status.observedDomains reflects every hostname
resolved cluster-wide, regardless of this object's namespace or podSelector. Real per-pod
attribution would need either a custom CoreDNS build (the edns0 plugin, which carries the
real client as an EDNS Client-Subnet option, but isn't compiled into the standard
registry.k8s.io/coredns/coredns image) or a different interception mechanism entirely
(eBPF/iptables at the source pod) — both out of scope here.
Requires --enable-snoop-resolver=true (see SnoopResolver setup); the
controller sets a Degraded condition when the snoop resolver isn't active. observedDomains
accumulates additively across reconciles, so it survives controller restarts.
Note
Out of scope: auto-generating an FQDNNetworkPolicy from the observation, opening a Git PR
— both are follow-on features that depend on this data being available.
A compromised dependency, a misconfigured app, or a supply-chain issue can cause a pod to start
resolving a hostname no policy covers — and until now the only signal was a dropped connection in
CNI logs. When the snoop resolver is active, every FQDNNetworkPolicy now compares its
namespace's combined egress rules against every hostname observed cluster-wide and reports the
ones nothing in the namespace covers:
kubectl get fqdnnetworkpolicy checkout -n payments -o jsonpath='{.status.observedUnpoliciedDomains}' | jq[
{ "hostname": "telemetry.vendor.com", "firstSeen": "2026-09-07T09:00:00Z", "lastSeen": "2026-09-07T14:22:00Z" }
]A newly-observed drifted hostname is logged (egress drift detected namespace=payments domain=telemetry.vendor.com) and counted via fqdnnp_unpolicied_domain_total{namespace,domain}.
The field is recomputed fresh every reconcile rather than accumulated, so it clears on its own
once a policy in the namespace covers the hostname. No behavior change when the snoop resolver is
inactive.
Warning
Same cluster-wide caveat as FQDNEgressObservation above applies here too: "unpolicied"
means "not covered by any FQDNNetworkPolicy in this namespace," not "queried by a pod in
this namespace" — the underlying observation still can't be attributed to a source pod. If
two FQDNNetworkPolicy objects share a namespace, each recomputes the same diff independently,
so a single newly-drifting hostname can be logged/counted once per object rather than exactly
once.
Note
Out of scope: automatic egress blocking. There's no reliable way to infer which pod selector to lock down from a bare hostname observation, so this controller does not attempt it — doing so blindly could just as easily block legitimate traffic from a pod nothing was ever wrong with. Instead, review the suggested policy yourself:
kubectl fqdn-policy suggest payments/checkout > suggested-policy.yaml
# review it, then:
kubectl apply -f suggested-policy.yamlThis reads status.observedUnpoliciedDomains and prints a starting-point FQDNNetworkPolicy —
in Audit mode, so it has no effect until you review it and flip spec.mode to Enforce.
Nothing is written to the cluster by this command.
| Field | Type | Default | Description |
|---|---|---|---|
spec.mode |
Enforce | Audit |
Enforce |
Enforce writes a NetworkPolicy. Audit logs resolved IPs and fires a Kubernetes Event without writing any policy — safe for validating before enforcement. |
spec.podSelector.podSelector |
LabelSelector |
— | Pods this policy applies to. Empty selects all pods in the namespace. |
spec.egress[].match |
string | — | Hostname to allow. Plain FQDNs always work. Wildcards (*.example.com) require --enable-snoop-resolver. |
spec.egress[].ports[].port |
int32 | — | Destination port (1–65535). |
spec.egress[].ports[].protocol |
TCP | UDP | SCTP |
TCP |
Transport protocol. |
spec.resolutionTTLOverride |
int32 | — | Re-resolution interval in seconds (5–300). Overrides DNS TTLs. Useful for sources with unreliable TTL values. |
spec.coreDNSAddress |
string | — | host:port of cluster CoreDNS (e.g. 10.96.0.10:53). Queries CoreDNS rather than the node resolver to reduce internal DNS divergence. |
spec.security.maxCNAMEDepth |
int32 | no limit | Raises a Degraded condition when a hostname's CNAME chain exceeds this depth. |
spec.security.blockPrivateIPs |
bool | true |
Drops RFC1918/CGNAT addresses from the resolved allow-list. |
spec.security.blockLoopback |
bool | true |
Drops loopback addresses from the resolved allow-list. |
spec.security.blockLinkLocal |
bool | true |
Drops link-local addresses from the resolved allow-list. |
spec.security.blockOnDivergence |
bool | false |
Freezes the applied NetworkPolicy for a reconcile cycle rather than admitting a divergent IP set, when any resolved host's resolverDivergence > 0. See Resolver disagreement. |
spec.security.minResolverAgreement |
int32 | 1 (no filtering) |
Consensus mode: an IP is allow-listed only once at least this many upstream resolvers agree on it. No-op without per-resolver result data (multi-resolver/snoop resolver). See Resolver disagreement. |
All fields from FQDNNetworkPolicy plus:
| Field | Type | Description |
|---|---|---|
spec.namespaceSelector |
LabelSelector |
Namespaces to target. Empty selects all namespaces. |
| Field | Type | Default | Description |
|---|---|---|---|
spec.podSelector.podSelector |
LabelSelector |
— | Accepted for forward compatibility; not yet enforced — see Discovering egress domains. |
spec.observationWindow |
duration | 24h |
How long to accumulate before status.observationComplete is set. |
kubectl get fqdnnetworkpolicy <name> -o jsonpath='{.status.conditions}' | jq| Condition | Status | Reason | Meaning |
|---|---|---|---|
Ready |
True |
Reconciled |
NetworkPolicy is up to date. |
Ready |
False |
ResolutionDegraded |
One or more hosts failed to resolve. Stale IPs are kept — access is not revoked on a transient DNS error. |
Ready |
True |
AuditMode |
Audit mode is active; no NetworkPolicy was written. |
Ready / Degraded |
False / True |
DivergenceBlocked |
spec.security.blockOnDivergence is set and resolvers disagreed this cycle; the previously-applied NetworkPolicy was kept as-is. |
Degraded |
True |
ResolutionErrors |
Companion to Ready=False; carries error detail. |
ResolverDivergence |
True |
ResolverDisagreement |
At least one resolved host's upstream resolvers disagreed on its IPs this cycle. Message names each affected hostname. See Resolver disagreement. |
ResolverDivergence |
False |
NoDivergence |
All resolvers agreed on every resolved hostname this cycle. |
kubectl get events -n <namespace> --field-selector involvedObject.name=<policy-name>Events are published for: NetworkPolicyCreated, NetworkPolicyUpdated, ResolutionFailed,
AuditResolved, SyncFailed.
The controller exposes 18 metrics at :8080/metrics. Key metrics:
| Metric | Type | Description |
|---|---|---|
fqdn_dns_lookup_duration_seconds |
Histogram | DNS resolution latency by domain and resolver. |
fqdn_dns_lookup_failures_total |
Counter | Resolution failures by domain and error type (nxdomain, timeout, servfail, other). |
fqdn_dns_cache_hits_total |
Counter | Resolutions served from the snoop IP cache. |
fqdn_dns_snoop_observations_total |
Counter | DNS responses intercepted by the proxy, by query type. |
fqdn_ttl_expiry_lag_seconds |
Histogram | How late (in seconds) a domain was re-resolved after TTL expiry. |
fqdn_ip_changes_total |
Counter | Times the IP set for a domain changed, triggering a NetworkPolicy update. |
fqdn_networkpolicy_sync_duration_seconds |
Histogram | Time to apply a NetworkPolicy (create/update/skip). |
fqdn_managed_policies_total |
Gauge | Active FQDNNetworkPolicy objects. |
fqdn_cluster_managed_namespaces_total |
Gauge | Namespaces receiving policies from ClusterFQDNNetworkPolicy. |
fqdn_reconcile_duration_seconds |
Histogram | End-to-end reconcile time per controller. |
fqdnnp_asn_change_total |
Counter | ASN/org enrichment changes between cycles, by hostname/previous/current ASN (opt-in enricher only). |
fqdnnp_resolver_divergence_total |
Counter | Times a resolved hostname showed resolver disagreement, by hostname/policy. See Resolver disagreement. |
fqdnnp_unpolicied_domain_total |
Counter | Times a newly-observed hostname was found uncovered by any policy in its namespace, by namespace/domain. See Detecting egress drift. |
Enable a ServiceMonitor for prometheus-operator:
metrics:
serviceMonitor:
enabled: true
interval: 30s# Check what IPs are in the current policy
kubectl get networkpolicy fqdnnp-allow-payment-apis -n payments -o yaml
# Check resolution status
kubectl get fqdnnetworkpolicy allow-payment-apis -n payments -o wide
# Detailed status including resolved IPs and TTLs
kubectl get fqdnnetworkpolicy allow-payment-apis -n payments \
-o jsonpath='{.status.resolvedHosts}' | jqThe ValidatingAdmissionWebhook catches invalid policies at kubectl apply time rather than
surfacing errors only through status conditions.
What it validates:
matchis a valid FQDN or wildcard FQDN (*.example.com)- Wildcard rules are rejected when
--enable-snoop-resolveris not set - Duplicate
matchrules within a policy - Port numbers are in range 1–65535
- Protocol is one of
TCP,UDP,SCTP
The webhook server starts automatically on --webhook-bind-address (default :9443). Register
it with the Kubernetes API server by applying a ValidatingWebhookConfiguration pointing to the
controller service. For TLS, use cert-manager with the Helm chart's webhook.certManager values.
For production deployments, run at least 2 replicas with leader election enabled:
helm upgrade fqdn-network-policy charts/fqdn-network-policy/ \
--set replicaCount=2 \
--set leaderElection.enabled=trueOnly the leader reconciles resources and issues DNS queries. Standby replicas become the leader
instantly if the current leader's pod is evicted or fails. The leader election lease is stored
in kube-system under the key fqdn-network-policy.netsec.kunal.dev.
- Run 2+ replicas with
--leader-elect=true - Configure
--enable-multi-resolver=true(default) or--enable-snoop-resolver=true - Set
--coredns-addressto your cluster's DNS service IP - Apply a
ValidatingWebhookConfigurationpointing to the controller service with TLS - Enable
metrics.serviceMonitorif using prometheus-operator - Set resource
requestsandlimitsin values (provided defaults are conservative) - Confirm pod anti-affinity is scheduling replicas on different nodes (included in chart)
- Run
kubectl apply -f policy.yaml --dry-run=serverbefore enforcing new policies in production - Use
spec.mode: Auditto validate a new policy before switching toEnforce
api/v1alpha1/
fqdnnetworkpolicy_types.go Namespaced CRD: FQDNNetworkPolicy
clusterfqdnnetworkpolicy_types.go Cluster-scoped CRD: ClusterFQDNNetworkPolicy
fqdnegressobservation_types.go Namespaced CRD: FQDNEgressObservation (status-only)
zz_generated.deepcopy.go Generated — do not edit by hand
internal/dns/
resolver.go ActiveResolver: TTL-aware A+AAAA via miekg/dns
multi_resolver.go MultiResolver: union of 4 public upstreams
snoop_resolver.go SnoopResolver: DNS forwarding proxy (records IPs)
wildcard_resolver.go WildcardResolver: prefix expansion for *.foo.com
ip_cache.go IPCache: thread-safe TTL-respecting IP store
ttl_queue.go TTLQueue: min-heap for exact TTL scheduling
observation_store.go ObservationStore: cluster-wide hostname observations
internal/enrich/
client.go IPInfoClient: ASN/org lookups against ipinfo.io
cache.go 6h TTL cache (also the per-IP rate limit)
manager.go Manager: async worker pool, never blocks reconciliation
internal/controller/
fqdnnetworkpolicy_controller.go Namespace-scoped reconcile loop
clusterfqdnnetworkpolicy_controller.go Cluster-scoped reconcile + fanout
fqdnegressobservation_controller.go Cluster-wide DNS observation reconcile loop
enrichment.go Wires internal/enrich into ResolvedHost status
resolver_divergence.go ResolverDivergence condition + opt-in block/consensus helpers
drift.go ObservedUnpoliciedDomains diff against namespace policies
internal/netpol/
builder.go Converts resolved IPs → NetworkPolicy
helpers.go hostCIDR: /32 IPv4, /128 IPv6
internal/webhook/
validate.go ValidatingAdmissionWebhook HTTP handler
internal/metrics/
metrics.go 18 Prometheus metrics
cmd/main.go Manager setup, flag parsing, resolver wiring
cmd/kubectl-fqdn_policy/main.go kubectl plugin: preview + diff subcommands
charts/fqdn-network-policy/ Helm chart
config/
crd/bases/ Generated CRD YAML
rbac/role.yaml ClusterRole + ClusterRoleBinding
manager/deployment.yaml Controller Deployment + snoop proxy Service
samples/ Example CRs
.github/workflows/ci.yml CI: lint → unit → build → KinD E2E → helm lint
make generate # regenerate zz_generated.deepcopy.go after type changes
make manifests # regenerate config/crd/bases/*.yaml and config/rbac/role.yaml
make build # compile ./bin/manager
make run # run against current kubeconfig context
go test -short ./... # unit tests (skips network-dependent tests)
go test -race ./... # full test suite including network testsWarning
Codegen note: do not hand-edit zz_generated.deepcopy.go or any file under
config/crd/bases/. Both are overwritten by make generate and make manifests. Commit
the generated output alongside any type changes.
| Limitation | Status |
|---|---|
| SnoopResolver requires CoreDNS forward configuration | Manual CoreDNS ConfigMap patch; automatic patching is planned |
| Wildcard expansion via prefix list may miss unusual subdomains | SnoopResolver learns new prefixes dynamically once active |
| No mTLS / SPIFFE identity-based policy | Use Istio or SPIRE alongside this controller for identity-based controls |
| No multi-cluster federation | Each cluster runs its own controller; shared policy distribution is not yet implemented |
| Webhook TLS requires cert-manager or manual cert provisioning | cert-manager integration is included in the Helm chart |
FQDNEgressObservation reports cluster-wide, not per-pod or per-namespace |
Fundamental limitation of CoreDNS's forward plugin, not a bug — see Discovering egress domains |
| ASN enricher depends on ipinfo.io reachability and its rate limits | Opt-in and disabled by default; results are cached 6h per IP to stay within free-tier limits |
status.observedUnpoliciedDomains can't attribute a hostname to a source pod, same as FQDNEgressObservation |
Same CoreDNS forward limitation — see Detecting egress drift. No automatic egress blocking is attempted for this reason; use kubectl fqdn-policy suggest for a human-reviewed starting point instead |
fqdnnp_unpolicied_domain_total can log/count a newly-drifted domain more than once |
Confirmed via demo.sh: a fast re-reconcile of a short-TTL host (or two FQDNNetworkPolicy objects sharing a namespace) can both read a status snapshot from before the other's write landed and both see the domain as "new." Not worth a shared dedup store for a metric whose job is "this is happening," not exact-once counting — see internal/controller/drift.go |
Apache 2.0. See LICENSE.