Add ability to skip loading kernel modules in antrea-agent #5754
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In order to support some specialized distributions, we may need to provide users with the ability to skip loading kernel modules. In particular, this is required to support Talos Linux (see antrea-io#5707). The Antrea Agent may try to load modules in 2 places: 1. in the install-cni initContainer: we try to load modules, mostly as a sanity check. If loading the openvswitch module fails, the container fails. 2. in the antrea-ovs container: this is outside of our direct control, but the ovs-ctl start script will try to load the openvswitch module if not detected. For install-cni, we introduce an environment variable, SKIP_LOADING_KERNEL_MODULES. If set, we do not run modprobe at all. For antrea-ovs, we introduce a new flag, `--skip-kmod`, to the start_ovs script. If provided, we ensure that ovs-ctl will not try to run modprobe, by replacing the ovs-kmod-ctl utility script by a no-op. To simplify usage, we introduce a new Helm configuration value, `agent.dontLoadKernelModules`. If set to true, we will take care of both configurations above. Note that even when skipping "explicit" Kernel module loading, the module will still be automatically loaded on the host when starting OVS if needed. This seems to be expected for recent Linux Kernel versions. With this change, Antrea can run on Talos Linux (confirmed with both the Docker and QEMU provisioners). As part of this change, we also introduce the `agent.antreaOVS.extraEnv` Helm value, to inject arbitrary environment variables in the antrea-ovs container. This is for parity with other antrea-agent containers, and is not strictly required. Signed-off-by: Antonin Bas <abas@vmware.com>
tnqn
reviewed
Nov 30, 2023
| # skip-kmod flag, which prevents the ovs-ctl script from trying to load any Kernel module. In | ||
| # order for this to work, we need to turn ovs-kmod-ctl into a "no-op". | ||
| cp /usr/share/openvswitch/scripts/ovs-kmod-ctl /usr/share/openvswitch/scripts/ovs-kmod-ctl.bak | ||
| echo ":" > /usr/share/openvswitch/scripts/ovs-kmod-ctl |
There was a problem hiding this comment.
Does : has any special meaning as a script?
There was a problem hiding this comment.
It should be no-different from an empty script. I used : to emphasize that it was a shell script and that we meant for it to be a no-op.
| | agent.antreaOVS.logFileMaxNum | int | `4` | Max number of log files. | | ||
| | agent.antreaOVS.logFileMaxSize | int | `100` | Max size in MBs of any single log file. | | ||
| | agent.antreaOVS.resources | object | `{"requests":{"cpu":"200m"}}` | Resource requests and limits for the antrea-ovs container. | | ||
| | agent.antreaOVS.securityContext.capabilities | list | `["SYS_NICE","NET_ADMIN","SYS_ADMIN","IPC_LOCK"]` | Capabilities for the antrea-ovs container. | | ||
| | agent.antreaOVS.securityContext.privileged | bool | `false` | Run the antrea-ovs container as privileged. | | ||
| | agent.apiPort | int | `10350` | Port for the antrea-agent APIServer to serve on. | | ||
| | agent.dnsPolicy | string | `""` | DNS Policy for the antrea-agent Pods. If empty, the Kubernetes default will be used. | | ||
| | agent.dontLoadKernelModules | bool | `false` | Do not try to load any of the required Kernel modules (e.g., openvswitch) during initialization of the antrea-agent. Most users should never need to set this to true, but it may be required with some specific distributions. | |
There was a problem hiding this comment.
nit: why don't call it agent.loadKernelModules and defaults to true?
There was a problem hiding this comment.
I like having the default value be the default value for the type (boolean here) when possible. It shows that loading kernel modules is meant to be the default / general behavior here, and that we need to set a variable to change it.
| {{- if .Values.agent.dontLoadKernelModules }} | ||
| - name: SKIP_LOADING_KERNEL_MODULES | ||
| value: "1" | ||
| {{- end }} |
There was a problem hiding this comment.
should it stop mounting host-lib-modules in this case?
There was a problem hiding this comment.
that's a good point, we don't need it. The con is that it introduces one more difference between the 2 "modes", but that's not a big deal. I will make the change and run some sanity checks.
Signed-off-by: Antonin Bas <abas@vmware.com>
tnqn
added
the
action/release-note
|
/test-all |
|
Validated the latest version with a Talos cluster |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to support some specialized distributions, we may need to provide users with the ability to skip loading kernel modules. In particular, this is required to support Talos Linux (see #5707).
The Antrea Agent may try to load modules in 2 places:
For install-cni, we introduce an environment variable, SKIP_LOADING_KERNEL_MODULES. If set, we do not run modprobe at all.
For antrea-ovs, we introduce a new flag,
--skip-kmod, to the start_ovs script. If provided, we ensure that ovs-ctl will not try to run modprobe, by replacing the ovs-kmod-ctl utility script by a no-op.To simplify usage, we introduce a new Helm configuration value,
agent.dontLoadKernelModules. If set to true, we will take care of both configurations above.Note that even when skipping "explicit" Kernel module loading, the module will still be automatically loaded on the host when starting OVS if needed. This seems to be expected for recent Linux Kernel versions.
With this change, Antrea can run on Talos Linux (confirmed with both the Docker and QEMU provisioners).
As part of this change, we also introduce the
agent.antreaOVS.extraEnvHelm value, to inject arbitrary environment variables in the antrea-ovs container. This is for parity with other antrea-agent containers, and is not strictly required.