New container locator for docker/k8s on linux #5076
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The docker and k8s workload attestors work backwards from pid to container by inspecting the proc filesystem. Today, this happens by inspecting the cgroup file. Identifying the container ID (and pod UID) from the cgroup file has been a continual arms race. The k8s and docker workload attestors grew different mechanisms for trying to deal with the large variety in the output. Further, with cgroups v2 and private namespaces, the cgroup file might not have the container ID or pod UID information within it. This PR unifies the container ID (and pod UID) extraction for both the docker and k8s workload attestors. The new implementation searches the mountinfo file first for cgroups mounts. If not found, it will fall back to the cgroup file (typically necessary only when the workload is running in the same container as the agent). The extraction algorithm is the same for both mountinfo and cgroup entries, and is as follows: 1. Iterator over each entry in the file being searched, extracting either the cgroup mount root (mountinfo) or the cgroup group path (cgroup) as the source path. 2. Walk backwards through the segments in the source path looking for the 64-bit hex digit container ID. 3. If looking for the pod UID (K8s only), then walk backwards through the segments in the path looking for the pod UID pattern used by kubelet. Start with the segment the container ID was found in (truncated to remove the container ID portion). 4. If there are pod UID/container ID conflicts after searching these files then log and abort. Entries that have a pod UID override those that don't. The container ID is very often contained in the last segment in the path but there are situations where it isn't. This new functionality is NOT enabled by default, but opted in using the `use_new_container_locator` configurable in each plugin. In 1.10, we can consider enabling it by default. The testing for the new code is spread out a little bit. The cgroups fallback functionality is mostly tested by the existing tests in the k8s and docker plugin tests. The mountinfo tests are only in the new containerinfo package. In the long term, I'd like to see all of the container info extraction related tests moved solely to the containerinfo package and removed from the individual plugins. Resolves spiffe#4004, resolves spiffe#4682, resolves spiffe#4917. Signed-off-by: Andrew Harding <azdagron@gmail.com>
azdagron
requested review from
evan2645,
amartinezfayo,
MarcosDY and
rturner3
as code owners
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
MarcosDY
reviewed
Apr 22, 2024
zmt
reviewed
Apr 22, 2024
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
Signed-off-by: Andrew Harding <azdagron@gmail.com>
3 tasks
MarcosDY
approved these changes
Apr 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The docker and k8s workload attestors work backwards from pid to container by inspecting the proc filesystem. Today, this happens by inspecting the cgroup file. Identifying the container ID (and pod UID) from the cgroup file has been a continual arms race. The k8s and docker workload attestors grew different mechanisms for trying to deal with the large variety in the output.
Further, with cgroups v2 and private namespaces, the cgroup file might not have the container ID or pod UID information within it.
This PR unifies the container ID (and pod UID) extraction for both the docker and k8s workload attestors. The new implementation searches the mountinfo file first for cgroups mounts. If not found, it will fall back to the cgroup file (typically necessary only when the workload is running in the same container as the agent).
The extraction algorithm is the same for both mountinfo and cgroup entries, and is as follows:
The container ID is very often contained in the last segment in the path but there are situations where it isn't.
This new functionality is NOT enabled by default, but opted in using the
use_new_container_locatorconfigurable in each plugin. In 1.10, we can consider enabling it by default.The testing for the new code is spread out a little bit. The cgroups fallback functionality is mostly tested by the existing tests in the k8s and docker plugin tests. The mountinfo tests are only in the new containerinfo package.
In the long term, I'd like to see all of the container info extraction related tests moved solely to the containerinfo package and removed from the individual plugins.
Resolves #4004, resolves #4682, resolves #4917.