bpf: fix iptables masquerading for node -> remote pod traffic

[jibi]

When Cilium runs with KPR, host-firewall or bandwidth manager, it will
try to auto-derive one or more devices to which the bpf_host program is
attached.

This program will, among other things, redirect ingress traffic destined
to a pod to the pod's lxc device using bpf_redirect().

This causes the traffic to bypass the nf_conntrack table, leading to a
situation where traffic leaving the pod after the connection's been
established will be (incorrectly) masqueraded in case Iptables
masquerading is enabled, since the connection is not tracked by
netfilter.

This commit fixes this by skipping bpf_redirect() when we detect this
case (i.e. traffic is flowing through bpf_host attached to a physical
device and Cilium has installed Iptables rules which require conntrack).

Fixes: #14859.
Fixes: #12205.

[jibi]

test-me-please

[pchaigno]

Isn't the same fix required for the IPv6 path?

[jibi]

Isn't the same fix required for the IPv6 path?

Right, I forgot about that since in direct routing mode pods can only have v4 IPs assigned to (as native-routing-cidr is a v4 CIDR), but yeah, let's fix that path as well since we are on it

[brb]

since in direct routing mode pods can only have v4 IPs assigned to (as native-routing-cidr is a v4 CIDR)

In the IPv6 case the following used for the MASQ exclusion - https://github.com/cilium/cilium/blob/master/pkg/datapath/config.go#L141.

it will try to auto-derive a device to which the bpf_host program is attached.

Nit: It can derive more than one devices.

Anyway, it's getting more difficult to wrap my head around NO_REDIRECT usage and implications. I've created the issue for that #16166. I guess we need to fix your particular case anyway before doing my suggested change.

[jibi]

since in direct routing mode pods can only have v4 IPs assigned to (as native-routing-cidr is a v4 CIDR)

In the IPv6 case the following used for the MASQ exclusion - https://github.com/cilium/cilium/blob/master/pkg/datapath/config.go#L141.

it will try to auto-derive a device to which the bpf_host program is attached.

Nit: It can derive more than one devices.

Anyway, it's getting more difficult to wrap my head around NO_REDIRECT usage and implications. I've created the issue for that #16166. I guess we need to fix your particular case anyway before doing my suggested change.

I think ideally we should move most of this conditional logic to userspace, and have NO_REDIRECT being a single source of truth for when we actually need to skip redirect (without the need to check for encap vs direct routing or nodeport being enabled in the datapath)

[jibi]

test-me-please

[brb]

I think ideally we should move most of this conditional logic to userspace, and have NO_REDIRECT being a single source of truth for when we actually need to skip redirect (without the need to check for encap vs direct routing or nodeport being enabled in the datapath)

@jibi 👍 👍 👍