Skip to content

About

Kubernetes operator for FQDN-based NetworkPolicies — resolves hostnames to IPs and keeps egress rules in sync with DNS TTLs

Resources

Code of conduct

Contributing

Security policy

Stars

18 stars

Watchers

0 watching

Forks

fqdn-network-policy

FQDN-based egress control for Kubernetes, on any CNI.

Write egress rules against hostnames instead of hand-curated CIDR lists. The controller resolves them to IPs, tracks DNS TTLs, and reconciles a standard networking.k8s.io/v1 NetworkPolicy — so enforcement stays with whatever CNI you already run: Calico, AWS VPC CNI, Azure CNI, Flannel, or anything else that honors NetworkPolicy.

CI E2E KinD Go 1.26 License: Apache 2.0 kubectl plugin

apiVersion: netsec.kunal.dev/v1alpha1
kind: FQDNNetworkPolicy
metadata:
  name: allow-payment-apis
  namespace: payments
spec:
  mode: Enforce
  podSelector:
    podSelector:
      matchLabels:
        app: checkout
  egress:
    - match: api.stripe.com
      ports:
        - port: 443
          protocol: TCP
    - match: api.github.com
      ports:
        - port: 443
          protocol: TCP
Table of contents

Why this exists

Standard Kubernetes NetworkPolicy accepts only CIDR ipBlock rules. External APIs — Stripe, GitHub, S3, auth providers — rotate IPs continuously with short DNS TTLs, run behind CDNs with anycast routing, and return different IPs to different callers. Maintaining hand-curated CIDR lists is error-prone and breaks silently. This controller keeps those lists current automatically.

How it compares

Feature fqdn-network-policy Cilium FQDNNetworkPolicy Calico FQDN policy Antrea FQDN Istio Egress
CNI requirement None — any CNI Cilium only Calico only Antrea only None
Sidecar required No No No No Yes (Envoy)
DNS resolution method Active + 4-resolver union + eBPF snoop eBPF kernel intercept DNS poll DNS proxy SNI inspection
Produces standard NetworkPolicy Yes No — Cilium CRD No — Calico CRD No — Antrea CRD No — VirtualService
CDN / anycast accuracy ~85 % (4 resolvers) Near-exact ~70 % Moderate SNI-based
Wildcard rules Prefix expansion / snoop Yes Yes Yes Yes
Cluster-wide policy Yes (ClusterFQDNNetworkPolicy) Yes Yes Yes Yes
kubectl preview plugin Yes — kubectl fqdn-policy preview No No No No
DNS rebinding protection Yes — blockPrivateIPs default Partial No No N/A
CNAME chain visibility Yes — surfaced in status No No No No
Multi-upstream resolver Yes — unions Cloudflare, Google, Quad9, OpenDNS No No No No
Infrastructure additions None Replace CNI Replace CNI Replace CNI Service mesh

Tip

Choose Cilium if you need exact per-pod DNS accuracy at scale and can replace your CNI. Choose this project if you need portable egress control on the CNI you already run, with zero infrastructure additions.

When to use this instead of Cilium or Istio:

  • You run Calico, AWS VPC CNI, Azure CNI, or Flannel and cannot or do not want to replace your CNI.
  • You need lightweight egress control with zero infrastructure additions (no sidecars, no service mesh, no CNI replacement).
  • Your egress targets are stable internal APIs, corporate auth providers, or fixed SaaS endpoints.

When to use Cilium instead: if you need wildcard accuracy for CDN-backed services (Cloudflare, Fastly, AWS CloudFront) at production scale, Cilium's eBPF DNS snooping is the correct tool. This controller includes a multi-upstream resolver that covers ~85% of CDN divergence cases but is not equivalent to in-kernel packet interception.


Architecture

flowchart TD
    YAML["FQDNNetworkPolicy /<br/>ClusterFQDNNetworkPolicy YAML"] -->|kubectl apply| Webhook

    subgraph CP[Control plane]
        Webhook["Admission webhook<br/>syntax, wildcards, ports"]
        Controller["Controller<br/>reconcile loop"]
        Webhook --> Controller
    end

    subgraph RES[Resolution]
        Resolver["Resolver<br/>Active / Multi / CoreDNS / Snoop"]
        Filter["Security filter<br/>private IPs, consensus, divergence"]
        Builder["NetworkPolicy builder"]
        Resolver --> Filter --> Builder
    end

    Controller --> Resolver
    Builder --> NP["networking.k8s.io/v1<br/>NetworkPolicy"]

    subgraph DNS[DNS path]
        CoreDNS["CoreDNS"]
        Snoop["SnoopResolver proxy"]
        Upstream["Upstream resolvers"]
        CoreDNS -->|forward| Snoop --> Upstream
    end

    Resolver -->|active lookups| Upstream
    Snoop -.->|observed IPs + hostnames| Resolver

    subgraph DP[Data plane]
        CNI["CNI<br/>Calico, AWS VPC CNI, Azure CNI, ..."]
        Pod["Workload pod"]
        Pod -->|DNS query| CoreDNS
    end

    NP --> CNI
    CNI ==>|egress enforced| Pod

    classDef hot fill:#1f6feb,color:#fff,stroke:#1f6feb;
    class Controller,Builder hot
Loading

Note

Fail-closed: if no hosts have resolved yet (first reconcile, cold start, all lookups failed), the controller emits a deny-all-egress policy rather than a permissive no-op. DNS egress (UDP/TCP port 53) is always permitted in the generated policy so resolution can continue.

Transient DNS safety: a resolution failure retains the last known-good IPs for that host rather than revoking access. The Degraded status condition is set while this fallback is active.


kubectl plugin

kubectl fqdn-policy lets you inspect what a policy will do before applying it.

Install

make kubectl-plugin
cp bin/kubectl-fqdn_policy /usr/local/bin/

preview — resolve without touching the cluster

kubectl fqdn-policy preview policy.yaml
Policy:    allow-payment-apis
Namespace: payments

api.stripe.com
  A:     3.18.12.10, 3.18.12.11
  AAAA:  2600:1f18:2148:bc01::a
  TTL:   38s
  CNAME: (none)
  Resolver divergence: 4 IPs across 4 resolvers

Generated NetworkPolicy would allow:
  3.18.12.10/32:443/TCP
  3.18.12.11/32:443/TCP
  2600:1f18:2148:bc01::a/128:443/TCP

Warnings:
  [WARN] api.stripe.com: resolver divergence (4 IPs appeared in fewer than all resolvers)

Flags:

  • --coredns-address <host:port> — query cluster CoreDNS instead of public resolvers
  • --no-multi-resolver — use the system resolver only

diff — compare cluster state against fresh resolution

kubectl fqdn-policy diff payments/allow-payment-apis
# or
kubectl fqdn-policy diff allow-payment-apis -n payments
Policy: allow-payment-apis (namespace: payments)

api.stripe.com
  + 54.160.0.1/32  (new)
  - 3.18.12.11/32  (stale)

Warnings shown for: private / loopback / link-local IPs, resolver divergence, CNAME chain depth > 3.

suggest — draft a policy from observed egress drift

kubectl fqdn-policy suggest payments/checkout

Reads the named FQDNNetworkPolicy's status.observedUnpoliciedDomains (see Detecting egress drift) and prints a suggested FQDNNetworkPolicy manifest — one rule per drifted hostname, starting in Audit mode — for you to review, edit, and kubectl apply -f - yourself. Nothing is written to the cluster by this command.

profile — discover and synthesize policies from traffic

Synthesizes a production-ready FQDNNetworkPolicy from observed DNS traffic (FQDNEgressObservation or an offline observation file). Filters cluster-internal DNS names (.cluster.local, arpa, localhost), clusters high-cardinality subdomains into wildcards (*.stripe.com), classifies by provider/category, and applies safe security defaults in Audit mode:

# Discover from live cluster observation
kubectl fqdn-policy profile cluster-egress-observation -n payments --app-label app=checkout

# Run offline / air-gapped from an exported observation file
kubectl fqdn-policy profile --from-file observation.yaml --wildcard-threshold 3 --preset strict

Flags:

  • --from-observation <name> — Name of the FQDNEgressObservation resource in cluster
  • --from-file <path> — Offline YAML/JSON observation file (works in air-gapped / CI environments)
  • --wildcard-threshold <int> — Subdomains required under the same domain to trigger wildcard rollup (default: 3, 0 to disable)
  • --preset <production|strict|relaxed> — Security preset (default: production with blockPrivateIPs: true)
  • --app-label <key=val> — Set podSelector.matchLabels (e.g. app=checkout)
  • --mode <Audit|Enforce> — Policy mode (default: Audit)
  • -o, --output <file> — Write policy YAML to file instead of stdout

simulate — evaluate blast radius before enforcing

Performs pre-flight blast radius simulation of a proposed policy against real observed cluster DNS queries (from an active observation or an offline baseline file). Identifies exactly which domains will be permitted and which domains will be BLOCKED, preventing production outages before switching from Audit to Enforce:

# Simulate policy against recorded baseline
kubectl fqdn-policy simulate policy.yaml --baseline observation.yaml

# CI/CD Gate: Fail build if any active domain would be blocked
kubectl fqdn-policy simulate payments/checkout-policy --fail-on-blocked

# Machine-readable JSON output for automated GitOps pipelines
kubectl fqdn-policy simulate policy.yaml --baseline observation.yaml --json

What the simulation checks:

  • Blast Radius Analysis: Exact list of actively queried hostnames that are missing from the policy and would be blocked.
  • Coverage Breakdown: Total domains, permitted domains, dropped domains, and ignored internal Kubernetes services.
  • Security & Sanity Audits: Flags empty egress rules, overly broad wildcards (*, *.com), and disabled private IP blocking.

Installation

Helm (recommended)

helm install fqdn-network-policy charts/fqdn-network-policy/ \
  --namespace fqdn-network-policy-system \
  --create-namespace \
  --set replicaCount=2 \
  --set leaderElection.enabled=true \
  --set multiResolver.enabled=true

Verify the controller is running:

kubectl get pods -n fqdn-network-policy-system
kubectl get fqdnnetworkpolicies --all-namespaces

Manual (without Helm)

# Install CRDs
kubectl apply -f config/crd/bases/

# Install RBAC and controller
kubectl apply -f config/rbac/role.yaml
kubectl apply -f config/manager/deployment.yaml

Local development

make build          # compile the manager binary to ./bin/manager
make run            # run against your current kubeconfig context
./demo.sh           # full KinD demo with Calico enforcement proof

Prerequisites: kind, kubectl, docker on PATH.


Writing policies

Namespace-scoped: FQDNNetworkPolicy

Applies to pods in a single namespace.

apiVersion: netsec.kunal.dev/v1alpha1
kind: FQDNNetworkPolicy
metadata:
  name: allow-external-apis
  namespace: payments
spec:
  # mode: Enforce writes a NetworkPolicy. mode: Audit logs only — safe for validation.
  mode: Enforce

  podSelector:
    podSelector:
      matchLabels:
        app: checkout

  egress:
    - match: api.stripe.com
      ports:
        - port: 443
          protocol: TCP
    - match: api.github.com
      ports:
        - port: 443
          protocol: TCP

    # Wildcards require --enable-snoop-resolver=true (see Resolver Strategy below)
    - match: "*.s3.amazonaws.com"
      ports:
        - port: 443
          protocol: TCP

  # Optional: override DNS TTL-based scheduling. Min 5s, max 300s.
  # resolutionTTLOverride: 30

  # Optional: resolve against cluster CoreDNS instead of the node resolver.
  # coreDNSAddress: "10.96.0.10:53"

Cluster-scoped: ClusterFQDNNetworkPolicy

Enforces baseline egress rules across all matched namespaces. Platform teams use this for corporate telemetry, auth providers, and shared infrastructure without duplicating CRs per namespace.

apiVersion: netsec.kunal.dev/v1alpha1
kind: ClusterFQDNNetworkPolicy
metadata:
  name: allow-corporate-telemetry
spec:
  mode: Enforce

  # Apply to all namespaces labelled env=production
  namespaceSelector:
    matchLabels:
      env: production

  podSelector:
    podSelector: {}   # all pods in matched namespaces

  egress:
    - match: otel-collector.corp.example.com
      ports:
        - port: 4317
          protocol: TCP
    - match: auth.corp.example.com
      ports:
        - port: 443
          protocol: TCP

The controller creates one NetworkPolicy per matched namespace (named cfqdnnp-<policy-name>) and garbage-collects it if the namespace no longer matches the selector.


Resolver strategy

Configure the resolver that best matches your infrastructure:

Resolver Flag CDN accuracy Wildcard Privilege required
SnoopResolver --enable-snoop-resolver Exact — sees actual pod responses Yes (dynamic) CoreDNS forward config
MultiResolver --enable-multi-resolver (default) ~85% — unions 4 public resolvers Yes (prefix expansion) None
CoreDNS direct --coredns-address Better for internal split-horizon No None
ActiveResolver (fallback) Same as node resolver No None

SnoopResolver setup

The SnoopResolver runs a DNS forwarding proxy that CoreDNS routes queries through. It records exactly which IPs pods received in its response stream.

  1. Start the controller with snoop enabled:

    --enable-snoop-resolver=true
    --snoop-listen-address=0.0.0.0:5353
    --snoop-upstream=<a real upstream, e.g. 8.8.8.8:53>   # never point this back at CoreDNS
  2. Patch your CoreDNS ConfigMap to forward through the snoop proxy. forward only accepts an IP address (or /etc/resolv.conf), not a Service DNS name — use the snoop Service's ClusterIP:

    .:53 {
        forward . <snoop-service-cluster-ip>:5353
        cache 30
        reload
    }
    
  3. Wildcard rules (e.g. *.s3.amazonaws.com) are now accepted by the admission webhook and dynamically expanded as the proxy observes new subdomains from pod traffic.

  4. The snoop resolver is also what backs FQDNEgressObservation below — but note its cluster-wide (not per-pod) limitation before relying on it for discovery.

MultiResolver (default, recommended)

Queries Cloudflare (1.1.1.1), Google (8.8.8.8), Quad9 (9.9.9.9), and OpenDNS (208.67.222.222) concurrently. Returns the union of all A and AAAA records. For CDN-backed services that use anycast routing, different resolvers return different PoP IPs — unioning them covers ~85% of divergence cases with no additional infrastructure.

Enable in Helm:

multiResolver:
  enabled: true   # default
coreDNS:
  address: "10.96.0.10:53"   # also queries cluster CoreDNS when set

Resolver disagreement

MultiResolver unions answers from several upstreams, which is exactly the right default for CDN anycast — but a large IP set can also mean something worth a second look (split-horizon leak, resolver poisoning, a geo-DNS oddity), and unioning silently threw away which resolver said what. When resolvers disagree on a hostname, that's now visible:

kubectl get fqdnnetworkpolicy allow-payment-apis -n payments -o yaml
status:
  resolvedHosts:
    - hostname: api.stripe.com
      security:
        resolverDivergence: 2
        resolverResults:            # only present when resolverDivergence > 0
          1.1.1.1:53: ["3.1.4.1", "3.1.4.2"]
          8.8.8.8:53: ["3.1.4.1"]
  conditions:
    - type: ResolverDivergence
      status: "True"
      reason: ResolverDisagreement
      message: "api.stripe.com: 2 resolvers returned 2 IPs total, 1 IPs appeared in only 1 resolver"

fqdnnp_resolver_divergence_total{hostname,policy} tracks the same signal in Prometheus.

Opt-in follow-ons, both off by default in spec.security:

  • blockOnDivergence — freeze the currently-applied NetworkPolicy for a reconcile cycle instead of admitting a divergent IP set. Status still updates (so you can see why), but no NetworkPolicy write happens until the next clean resolution.
  • minResolverAgreement — consensus mode. An IP only makes it into the allow-list once at least this many upstream resolvers agree on it. Requires per-resolver result data (MultiResolver or SnoopResolver); a no-op with a plain ActiveResolver/CoreDNSResolver.

Neither changes the default union behavior unless you turn them on.


ASN/org enrichment (opt-in)

Annotate resolved IPs with autonomous-system and organization data, so an operator can tell at a glance whether api.stripe.com resolving to an unfamiliar IP is a new CDN edge (expected) or an unrelated hosting provider (suspicious). Disabled by default — no external calls unless configured.

--asn-enricher=ipinfo      # or FQDNNP_ASN_ENRICHER=ipinfo
--ipinfo-token=<token>     # optional, or FQDNNP_IPINFO_TOKEN; raises ipinfo.io rate limits

When enabled, each resolved IP is looked up against ipinfo.io asynchronously — enrichment never blocks policy reconciliation. Results are cached for 6 hours per IP (also the enrichment rate limit) and populate status.resolvedHosts[].ipEnrichments:

kubectl get fqdnnetworkpolicy allow-payment-apis -n payments \
  -o jsonpath='{.status.resolvedHosts[0].ipEnrichments}' | jq
{
  "54.160.0.1": {
    "asn": "AS16509",
    "org": "Amazon.com, Inc.",
    "country": "US",
    "enrichedAt": "2026-09-07T08:39:52Z"
  }
}

If an IP's ASN/org changes between enrichment cycles (e.g. a hostname moves to a different hosting provider), the controller logs a warning and increments fqdnnp_asn_change_total. Blocking policy application on an ASN change, and allowlisting by expected ASN, are both follow-on features — this only surfaces the data.


Discovering egress domains: FQDNEgressObservation

Writing an FQDNNetworkPolicy from scratch requires knowing what hostnames your application resolves. FQDNEgressObservation surfaces that passively from real DNS traffic, via the SnoopResolver — no code changes, no guessing, no watching for connection failures after applying a policy.

apiVersion: netsec.kunal.dev/v1alpha1
kind: FQDNEgressObservation
metadata:
  name: checkout-observation
  namespace: payments
spec:
  podSelector:
    podSelector:
      matchLabels:
        app: checkout
  observationWindow: 24h   # how long to accumulate before status.observationComplete is set
kubectl get fqdnegressobservation checkout-observation -n payments \
  -o jsonpath='{.status.observedDomains}' | jq
[
  { "hostname": "api.stripe.com", "firstSeen": "2026-09-07T09:00:00Z", "lastSeen": "2026-09-07T14:22:00Z" },
  { "hostname": "api.github.com", "firstSeen": "2026-09-07T09:01:00Z", "lastSeen": "2026-09-07T14:22:00Z" }
]

Warning

This is cluster-wide, not per-pod. spec.podSelector is accepted for forward compatibility but has no filtering effect today: CoreDNS's forward plugin re-originates every forwarded query as its own client, so the snoop proxy can never see which pod actually asked — only that a query happened, somewhere in the cluster. status.observedDomains reflects every hostname resolved cluster-wide, regardless of this object's namespace or podSelector. Real per-pod attribution would need either a custom CoreDNS build (the edns0 plugin, which carries the real client as an EDNS Client-Subnet option, but isn't compiled into the standard registry.k8s.io/coredns/coredns image) or a different interception mechanism entirely (eBPF/iptables at the source pod) — both out of scope here.

Requires --enable-snoop-resolver=true (see SnoopResolver setup); the controller sets a Degraded condition when the snoop resolver isn't active. observedDomains accumulates additively across reconciles, so it survives controller restarts.

Note

Out of scope: auto-generating an FQDNNetworkPolicy from the observation, opening a Git PR — both are follow-on features that depend on this data being available.


Detecting egress drift

A compromised dependency, a misconfigured app, or a supply-chain issue can cause a pod to start resolving a hostname no policy covers — and until now the only signal was a dropped connection in CNI logs. When the snoop resolver is active, every FQDNNetworkPolicy now compares its namespace's combined egress rules against every hostname observed cluster-wide and reports the ones nothing in the namespace covers:

kubectl get fqdnnetworkpolicy checkout -n payments -o jsonpath='{.status.observedUnpoliciedDomains}' | jq
[
  { "hostname": "telemetry.vendor.com", "firstSeen": "2026-09-07T09:00:00Z", "lastSeen": "2026-09-07T14:22:00Z" }
]

A newly-observed drifted hostname is logged (egress drift detected namespace=payments domain=telemetry.vendor.com) and counted via fqdnnp_unpolicied_domain_total{namespace,domain}. The field is recomputed fresh every reconcile rather than accumulated, so it clears on its own once a policy in the namespace covers the hostname. No behavior change when the snoop resolver is inactive.

Warning

Same cluster-wide caveat as FQDNEgressObservation above applies here too: "unpolicied" means "not covered by any FQDNNetworkPolicy in this namespace," not "queried by a pod in this namespace" — the underlying observation still can't be attributed to a source pod. If two FQDNNetworkPolicy objects share a namespace, each recomputes the same diff independently, so a single newly-drifting hostname can be logged/counted once per object rather than exactly once.

Note

Out of scope: automatic egress blocking. There's no reliable way to infer which pod selector to lock down from a bare hostname observation, so this controller does not attempt it — doing so blindly could just as easily block legitimate traffic from a pod nothing was ever wrong with. Instead, review the suggested policy yourself:

kubectl fqdn-policy suggest payments/checkout > suggested-policy.yaml
# review it, then:
kubectl apply -f suggested-policy.yaml

This reads status.observedUnpoliciedDomains and prints a starting-point FQDNNetworkPolicy — in Audit mode, so it has no effect until you review it and flip spec.mode to Enforce. Nothing is written to the cluster by this command.


Spec reference

FQDNNetworkPolicy

Field Type Default Description
spec.mode Enforce | Audit Enforce Enforce writes a NetworkPolicy. Audit logs resolved IPs and fires a Kubernetes Event without writing any policy — safe for validating before enforcement.
spec.podSelector.podSelector LabelSelector — Pods this policy applies to. Empty selects all pods in the namespace.
spec.egress[].match string — Hostname to allow. Plain FQDNs always work. Wildcards (*.example.com) require --enable-snoop-resolver.
spec.egress[].ports[].port int32 — Destination port (1–65535).
spec.egress[].ports[].protocol TCP | UDP | SCTP TCP Transport protocol.
spec.resolutionTTLOverride int32 — Re-resolution interval in seconds (5–300). Overrides DNS TTLs. Useful for sources with unreliable TTL values.
spec.coreDNSAddress string — host:port of cluster CoreDNS (e.g. 10.96.0.10:53). Queries CoreDNS rather than the node resolver to reduce internal DNS divergence.
spec.security.maxCNAMEDepth int32 no limit Raises a Degraded condition when a hostname's CNAME chain exceeds this depth.
spec.security.blockPrivateIPs bool true Drops RFC1918/CGNAT addresses from the resolved allow-list.
spec.security.blockLoopback bool true Drops loopback addresses from the resolved allow-list.
spec.security.blockLinkLocal bool true Drops link-local addresses from the resolved allow-list.
spec.security.blockOnDivergence bool false Freezes the applied NetworkPolicy for a reconcile cycle rather than admitting a divergent IP set, when any resolved host's resolverDivergence > 0. See Resolver disagreement.
spec.security.minResolverAgreement int32 1 (no filtering) Consensus mode: an IP is allow-listed only once at least this many upstream resolvers agree on it. No-op without per-resolver result data (multi-resolver/snoop resolver). See Resolver disagreement.

ClusterFQDNNetworkPolicy

All fields from FQDNNetworkPolicy plus:

Field Type Description
spec.namespaceSelector LabelSelector Namespaces to target. Empty selects all namespaces.

FQDNEgressObservation

Field Type Default Description
spec.podSelector.podSelector LabelSelector — Accepted for forward compatibility; not yet enforced — see Discovering egress domains.
spec.observationWindow duration 24h How long to accumulate before status.observationComplete is set.

Status and observability

Conditions

kubectl get fqdnnetworkpolicy <name> -o jsonpath='{.status.conditions}' | jq
Condition Status Reason Meaning
Ready True Reconciled NetworkPolicy is up to date.
Ready False ResolutionDegraded One or more hosts failed to resolve. Stale IPs are kept — access is not revoked on a transient DNS error.
Ready True AuditMode Audit mode is active; no NetworkPolicy was written.
Ready / Degraded False / True DivergenceBlocked spec.security.blockOnDivergence is set and resolvers disagreed this cycle; the previously-applied NetworkPolicy was kept as-is.
Degraded True ResolutionErrors Companion to Ready=False; carries error detail.
ResolverDivergence True ResolverDisagreement At least one resolved host's upstream resolvers disagreed on its IPs this cycle. Message names each affected hostname. See Resolver disagreement.
ResolverDivergence False NoDivergence All resolvers agreed on every resolved hostname this cycle.

Kubernetes Events

kubectl get events -n <namespace> --field-selector involvedObject.name=<policy-name>

Events are published for: NetworkPolicyCreated, NetworkPolicyUpdated, ResolutionFailed, AuditResolved, SyncFailed.

Prometheus metrics

The controller exposes 18 metrics at :8080/metrics. Key metrics:

Metric Type Description
fqdn_dns_lookup_duration_seconds Histogram DNS resolution latency by domain and resolver.
fqdn_dns_lookup_failures_total Counter Resolution failures by domain and error type (nxdomain, timeout, servfail, other).
fqdn_dns_cache_hits_total Counter Resolutions served from the snoop IP cache.
fqdn_dns_snoop_observations_total Counter DNS responses intercepted by the proxy, by query type.
fqdn_ttl_expiry_lag_seconds Histogram How late (in seconds) a domain was re-resolved after TTL expiry.
fqdn_ip_changes_total Counter Times the IP set for a domain changed, triggering a NetworkPolicy update.
fqdn_networkpolicy_sync_duration_seconds Histogram Time to apply a NetworkPolicy (create/update/skip).
fqdn_managed_policies_total Gauge Active FQDNNetworkPolicy objects.
fqdn_cluster_managed_namespaces_total Gauge Namespaces receiving policies from ClusterFQDNNetworkPolicy.
fqdn_reconcile_duration_seconds Histogram End-to-end reconcile time per controller.
fqdnnp_asn_change_total Counter ASN/org enrichment changes between cycles, by hostname/previous/current ASN (opt-in enricher only).
fqdnnp_resolver_divergence_total Counter Times a resolved hostname showed resolver disagreement, by hostname/policy. See Resolver disagreement.
fqdnnp_unpolicied_domain_total Counter Times a newly-observed hostname was found uncovered by any policy in its namespace, by namespace/domain. See Detecting egress drift.

Enable a ServiceMonitor for prometheus-operator:

metrics:
  serviceMonitor:
    enabled: true
    interval: 30s

Inspecting the generated NetworkPolicy

# Check what IPs are in the current policy
kubectl get networkpolicy fqdnnp-allow-payment-apis -n payments -o yaml

# Check resolution status
kubectl get fqdnnetworkpolicy allow-payment-apis -n payments -o wide

# Detailed status including resolved IPs and TTLs
kubectl get fqdnnetworkpolicy allow-payment-apis -n payments \
  -o jsonpath='{.status.resolvedHosts}' | jq

Admission webhook

The ValidatingAdmissionWebhook catches invalid policies at kubectl apply time rather than surfacing errors only through status conditions.

What it validates:

  • match is a valid FQDN or wildcard FQDN (*.example.com)
  • Wildcard rules are rejected when --enable-snoop-resolver is not set
  • Duplicate match rules within a policy
  • Port numbers are in range 1–65535
  • Protocol is one of TCP, UDP, SCTP

The webhook server starts automatically on --webhook-bind-address (default :9443). Register it with the Kubernetes API server by applying a ValidatingWebhookConfiguration pointing to the controller service. For TLS, use cert-manager with the Helm chart's webhook.certManager values.


High availability

For production deployments, run at least 2 replicas with leader election enabled:

helm upgrade fqdn-network-policy charts/fqdn-network-policy/ \
  --set replicaCount=2 \
  --set leaderElection.enabled=true

Only the leader reconciles resources and issues DNS queries. Standby replicas become the leader instantly if the current leader's pod is evicted or fails. The leader election lease is stored in kube-system under the key fqdn-network-policy.netsec.kunal.dev.


Production deployment checklist

  • Run 2+ replicas with --leader-elect=true
  • Configure --enable-multi-resolver=true (default) or --enable-snoop-resolver=true
  • Set --coredns-address to your cluster's DNS service IP
  • Apply a ValidatingWebhookConfiguration pointing to the controller service with TLS
  • Enable metrics.serviceMonitor if using prometheus-operator
  • Set resource requests and limits in values (provided defaults are conservative)
  • Confirm pod anti-affinity is scheduling replicas on different nodes (included in chart)
  • Run kubectl apply -f policy.yaml --dry-run=server before enforcing new policies in production
  • Use spec.mode: Audit to validate a new policy before switching to Enforce

Project layout

api/v1alpha1/
  fqdnnetworkpolicy_types.go           Namespaced CRD: FQDNNetworkPolicy
  clusterfqdnnetworkpolicy_types.go    Cluster-scoped CRD: ClusterFQDNNetworkPolicy
  fqdnegressobservation_types.go       Namespaced CRD: FQDNEgressObservation (status-only)
  zz_generated.deepcopy.go            Generated — do not edit by hand

internal/dns/
  resolver.go                          ActiveResolver: TTL-aware A+AAAA via miekg/dns
  multi_resolver.go                    MultiResolver: union of 4 public upstreams
  snoop_resolver.go                    SnoopResolver: DNS forwarding proxy (records IPs)
  wildcard_resolver.go                 WildcardResolver: prefix expansion for *.foo.com
  ip_cache.go                          IPCache: thread-safe TTL-respecting IP store
  ttl_queue.go                         TTLQueue: min-heap for exact TTL scheduling
  observation_store.go                 ObservationStore: cluster-wide hostname observations

internal/enrich/
  client.go                            IPInfoClient: ASN/org lookups against ipinfo.io
  cache.go                             6h TTL cache (also the per-IP rate limit)
  manager.go                           Manager: async worker pool, never blocks reconciliation

internal/controller/
  fqdnnetworkpolicy_controller.go      Namespace-scoped reconcile loop
  clusterfqdnnetworkpolicy_controller.go  Cluster-scoped reconcile + fanout
  fqdnegressobservation_controller.go  Cluster-wide DNS observation reconcile loop
  enrichment.go                        Wires internal/enrich into ResolvedHost status
  resolver_divergence.go               ResolverDivergence condition + opt-in block/consensus helpers
  drift.go                             ObservedUnpoliciedDomains diff against namespace policies

internal/netpol/
  builder.go                           Converts resolved IPs → NetworkPolicy
  helpers.go                           hostCIDR: /32 IPv4, /128 IPv6

internal/webhook/
  validate.go                          ValidatingAdmissionWebhook HTTP handler

internal/metrics/
  metrics.go                           18 Prometheus metrics

cmd/main.go                            Manager setup, flag parsing, resolver wiring
cmd/kubectl-fqdn_policy/main.go        kubectl plugin: preview + diff subcommands

charts/fqdn-network-policy/            Helm chart
config/
  crd/bases/                           Generated CRD YAML
  rbac/role.yaml                       ClusterRole + ClusterRoleBinding
  manager/deployment.yaml             Controller Deployment + snoop proxy Service
  samples/                             Example CRs

.github/workflows/ci.yml              CI: lint → unit → build → KinD E2E → helm lint

Development

make generate   # regenerate zz_generated.deepcopy.go after type changes
make manifests  # regenerate config/crd/bases/*.yaml and config/rbac/role.yaml
make build      # compile ./bin/manager
make run        # run against current kubeconfig context

go test -short ./...         # unit tests (skips network-dependent tests)
go test -race  ./...         # full test suite including network tests

Warning

Codegen note: do not hand-edit zz_generated.deepcopy.go or any file under config/crd/bases/. Both are overwritten by make generate and make manifests. Commit the generated output alongside any type changes.

Known limitations

Limitation Status
SnoopResolver requires CoreDNS forward configuration Manual CoreDNS ConfigMap patch; automatic patching is planned
Wildcard expansion via prefix list may miss unusual subdomains SnoopResolver learns new prefixes dynamically once active
No mTLS / SPIFFE identity-based policy Use Istio or SPIRE alongside this controller for identity-based controls
No multi-cluster federation Each cluster runs its own controller; shared policy distribution is not yet implemented
Webhook TLS requires cert-manager or manual cert provisioning cert-manager integration is included in the Helm chart
FQDNEgressObservation reports cluster-wide, not per-pod or per-namespace Fundamental limitation of CoreDNS's forward plugin, not a bug — see Discovering egress domains
ASN enricher depends on ipinfo.io reachability and its rate limits Opt-in and disabled by default; results are cached 6h per IP to stay within free-tier limits
status.observedUnpoliciedDomains can't attribute a hostname to a source pod, same as FQDNEgressObservation Same CoreDNS forward limitation — see Detecting egress drift. No automatic egress blocking is attempted for this reason; use kubectl fqdn-policy suggest for a human-reviewed starting point instead
fqdnnp_unpolicied_domain_total can log/count a newly-drifted domain more than once Confirmed via demo.sh: a fast re-reconcile of a short-TTL host (or two FQDNNetworkPolicy objects sharing a namespace) can both read a status snapshot from before the other's write landed and both see the domain as "new." Not worth a shared dedup store for a metric whose job is "this is happening," not exact-once counting — see internal/controller/drift.go

License

Apache 2.0. See LICENSE.

About

Kubernetes operator for FQDN-based NetworkPolicies — resolves hostnames to IPs and keeps egress rules in sync with DNS TTLs

Resources

Code of conduct

Contributing

Security policy

Stars

18 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages