Firewall Observability

[moomzni]

We've recently been experimenting with Kuberouter using the Azure Kubernetes Service and are finding if difficult to surface atypical iptables logging information including blocked requests; this makes it incredibly difficult to determine system traffic flows which need to be permitted or just debugging in general.

Perhaps I've missed the information in the readme/elsewhere, but it isn't clear to me what the recommended guidance is on how to debug network policies to understand what traffic is being blocked or just generally getting observability of the firewall behaviour so we can reason about its behaviour (to know that its working as expected between deployments)

[murali-reddy]

Sorry in current implementation observability of network policies is not really there. Other than observing iptable rule due to which packet is dropped there is no insights.

kube-router need to add support for logging. Possibly integration with https://netfilter.org/projects/ulogd/ is needed.

[murali-reddy]

#889 adds the functionality to log droped packets.

It works for the kernel above 4.11 you can configre host ulogd to view the dropeed packet
https://serverfault.com/questions/691730/iptables-log-rule-inside-a-network-namespace. However it does not work for kernel below 4.11 as logging from non-init network namespaces were deliberately dropped.

https://serverfault.com/questions/691730/iptables-log-rule-inside-a-network-namespace

As a future enhancement kube-router can load the log file (for e.g. /var/log/ulogd.pcap) and analyze to publish prometheus metrics and log in kube-router logs