bpf: Preserve overlay->lxc path with kube-proxy

The previous commit changed the packet handling on the path
overlay->lxc to fix a bug. More presicely, when endpoint routes are
enabled, we won't enforce ingress policies on both the overlay and the
lxc devices but only on the latter.

However, as a consequence of that patch, we don't go through the
policy-only program in bpf_lxc and we therefore changed the way the
packet is transmitted between overlay and lxc devices in some cases. As
a summary of changes made in the previous path, consider the following
table for the path overlay -> lxc.

Before the previous patch:
| Endpoint routes | Enforcement     | Path                 |
|-----------------|-----------------|----------------------|
| Enable          | overlay AND lxc | bpf_redirect if KPR; |
|                 |                 | stack otherwise      |
| Disabled        | overlay         | bpf_redirect         |

Now:
| Endpoint routes | Enforcement | Path         |
|-----------------|-------------|--------------|
| Enable          | lxc         | bpf_redirect |
| Disabled        | overlay     | bpf_redirect |

The previous patch intended to fix the enforcement to avoid the double
policy enforcement, but it also changed the packet path in case endpoint
routes are enabled.

This patch now fixes this by adding the same exception we have in
bpf_lxc to the l3.h logic we have. Hence, with the current patch, the
table will look like:
| Endpoint routes | Enforcement | Path                 |
|-----------------|-------------|----------------------|
| Enable          | lxc         | bpf_redirect if KPR; |
|                 |             | stack otherwise      |
| Disabled        | overlay     | bpf_redirect         |

I've kept this in a separate commit from the previous in an attempt to
split up and the logic and more clearly show the deltas.

Signed-off-by: Paul Chaignon <paul@cilium.io>

bpf/lib/l3.h

@@ -93,7 +93,19 @@ static __always_inline int ipv6_local_delivery(struct __ctx_buff *ctx, int l3_of
 	ctx->mark |= MARK_MAGIC_IDENTITY;
 	set_identity_mark(ctx, seclabel);
 
+# if defined(TUNNEL_MODE) && !defined(ENABLE_NODEPORT)
+	/* In tunneling mode, we execute this code to send the packet from
+	 * cilium_vxlan to lxc*. If we're using kube-proxy, we don't want to use
+	 * redirect() because that would bypass conntrack and the reverse DNAT.
+	 * Thus, we send packets to the stack, but since they have the wrong
+	 * Ethernet addresses, we need to mark them as PACKET_HOST or the kernel
+	 * will drop them.
+	 */
+	ctx_change_type(ctx, PACKET_HOST);
+	return CTX_ACT_OK;
+# else
 	return redirect_ep(ctx, ep->ifindex, from_host);
+# endif /* !ENABLE_ROUTING && TUNNEL_MODE && !ENABLE_NODEPORT */
 #else
 	/* Jumps to destination pod's BPF program to enforce ingress policies. */
 	ctx_store_meta(ctx, CB_SRC_LABEL, seclabel);
@@ -141,7 +153,19 @@ static __always_inline int ipv4_local_delivery(struct __ctx_buff *ctx, int l3_of
 	ctx->mark |= MARK_MAGIC_IDENTITY;
 	set_identity_mark(ctx, seclabel);
 
+# if defined(TUNNEL_MODE) && !defined(ENABLE_NODEPORT)
+	/* In tunneling mode, we execute this code to send the packet from
+	 * cilium_vxlan to lxc*. If we're using kube-proxy, we don't want to use
+	 * redirect() because that would bypass conntrack and the reverse DNAT.
+	 * Thus, we send packets to the stack, but since they have the wrong
+	 * Ethernet addresses, we need to mark them as PACKET_HOST or the kernel
+	 * will drop them.
+	 */
+	ctx_change_type(ctx, PACKET_HOST);
+	return CTX_ACT_OK;
+# else
 	return redirect_ep(ctx, ep->ifindex, from_host);
+# endif /* !ENABLE_ROUTING && TUNNEL_MODE && !ENABLE_NODEPORT */
 #else
 	/* Jumps to destination pod's BPF program to enforce ingress policies. */
 	ctx_store_meta(ctx, CB_SRC_LABEL, seclabel);