Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libnetwork: support encrypted overlay networks on systems without xt_u32 kernel module #45118

Closed
wants to merge 3 commits into from

Conversation

corhere
Copy link
Contributor

@corhere corhere commented Mar 7, 2023

The xt_u32 kernel module provides the xtables u32 match, which allows firewall rules to be written which match on the contents of the raw packet. The libnetwork overlay network driver creates firewall rules using u32 matches to match the VXLAN ID (VNI) of VXLAN datagrams so that the rules are selectively applied only to packets associated with VXLANs for the overlay networks managed by the driver. RHEL/CentOS 8 no longer ship the xt_u32 kernel module by default, and RHEL/CentOS 9 do not build it at all.

- What I did
I added an alternative implementation of the iptables rules to the overlay driver which does not depend on the xt_u32 kernel module. It can be selected at compile-time using the libnetwork_overlay_bpf build tag.

- How I did it
I wrote a BPF filter which is a drop-in replacement for the u32 expression and used that filter with the xt_bpf kernel module to match using the filter in the iptables rules.

- How to verify it
Spin up a Swarm cluster with at least two nodes, at least one of which is v23.0.1 (to test back-compatibility). Create an encrypted overlay network, start a container on each node attached to the network, and verify that processes on one node's container can connect to processes listening in the other node's container.

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

The overlay-network encryption code is woefully under-documented, which
is especially problematic as it operates on under-documented kernel
interfaces. Document what I have puzzled out of the implementation for
the benefit of the next poor soul to touch this code.

Signed-off-by: Cory Snider <csnider@mirantis.com>
The iptables rule clause used to match on the VNI of VXLAN datagrams
looks like line noise to the uninitiated. It doesn't help that the
expression is repeated twice and neither copy has any commentary.
DRY out the rule builder to a common function, and document what the
rule does and how it works.

Signed-off-by: Cory Snider <csnider@mirantis.com>
@corhere corhere added kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. area/networking labels Mar 7, 2023
@corhere corhere added this to the v-next milestone Mar 7, 2023
@corhere
Copy link
Contributor Author

corhere commented Mar 7, 2023

cc: @akerouanton

Some newer distros such as RHEL 9 have stopped making the xt_u32 kernel
module available with the kernels they ship. They do ship the xt_bpf
kernel module, which can do everything xt_u32 can and more. Add an
alternative implementation of the iptables match rule which uses xt_bpf
to implement exactly the same logic as the u32 filter using a BPF
program. Allow the implementation to be selected at compile time using a
build tag.

Signed-off-by: Cory Snider <csnider@mirantis.com>
@cpuguy83
Copy link
Member

cpuguy83 commented Mar 7, 2023

I thought we discussed switching on this at runtime?

@corhere
Copy link
Contributor Author

corhere commented Mar 7, 2023

This is an MVP to keep the review focus on the iptables rules themselves without getting sidetracked by runtime-switching concerns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/networking kind/enhancement Enhancements are not bugs or new features but can improve usability or performance.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants