Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
XDP not work well with Docker veth #3077
I wanna load XDP program to container’s veth.
My XDP program is as follows:
#define u32 unsigned int
SEC(“test_xdp”) int test_xdp_main(struct xdp_md *ctx)
char license SEC(“license”) = “GPL”;
which means all the arriving packets will be dropped.
Then, I compiled it with clang and loaded it to veth02b9ec2:
But when I ping 172.17.0.3 in host1 whose IP is 172.17.0.2, it seemed that XDP didn’t work at all. However, when I loaded this XDP program in docker0, and ping 172.17.0.1 or the address outside the container network at host1, the XDP program worked very well and ping failed.
My iproute2 version is ss180129, clang version is 4.0.0 and docker version is 1.12.6. I hope someone can help me out.
If I understand you correctly, you have two docker containers, host1 and host2. Inside host1 sits veth02b9ec2 with 172.17.0.2. Inside host2 container vethf2e33b9 with 172.17.0.3. You attach the drop-all XDP program to veth02b9ec2 in host1, and ping 172.17.0.2 -> 172.17.0.3. All correct? Now you are wondering that even though you've installed the XDP prog on veth02b9ec2 that nothing gets dropped.
Two reasons: i) XDP only enforces on ingress, meaning you're much better off attaching the XDP program onto the host facing veth device of host1 if you want to enforce policy for host1's egress. ii) While this might work for ping, there are cases where generic XDP fails today, in particular when the skb is cloned, since original use case for XDP generic was to mimic native XDP run on drivers for phys NICs. Meaning, the latter will fail on veth devices in case of TCP since it clones the packets. This can be fixed though, although at a bigger performance penalty.
The other option you of course have is to use tc/BPF on the veth devices instead (like Cilium does) and you can attach on ingress or egress via sch_clsact and cls_bpf with a very similar dropper prog. See also in Cilium's doc the BPF/XDP reference guide.
referenced this issue
Mar 12, 2018
I am hijacking this issue a bit since it still seems relevant.
Also are there any other kernel protocols where the skb is cloned beforehand? Interestingly, SCTP seems to work fine with XDP and virtual interfaces.
@fruffy here's something hacky that should unblock it, although it's very inefficient given this would call twice into pskb_expand_head() when skb is cloned and non-linear. Afaik, UDP generally doesn't do it, but TCP is the main one.