hostfw: scalability issue due to BPF connection tracking for overlay traffic

[julianwiedmann]

In the context of #31082, @qmonnet spotted that the Host Firewall code in to-netdev currently treats all overlay traffic as "host-originating", and thus creates CT entries for every VXLAN / GENEVE connection.

These CT entries are meant to enable policy-bypass for reply traffic - but due to how VXLAN / GENEVE work, this doesn't make sense (replies are addressed to the TUNNEL_PORT, not to the initial source port). Instead, the inbound path in from-netdev just creates another pair of CT entries.

All these additional CT entries can result in scalability concerns for the BPF conntrack map - potential LRU evictions, additional overhead when running GC on the map etc.

Proposal
For the outbound path, we could

1. transport a different identity, and completely bypass the HostFW path.
2. or use ctx_is_overlay() to apply policy, but then skip the CT entry creating.