Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shimpei to inject runc prestart hooks #1868

Merged
merged 2 commits into from Dec 14, 2021

Conversation

arnaldo2792
Copy link
Contributor

Issue number:
N / A

Description of changes:
Related to #1799 , that PR is quite big. I'm taking some of the commits in that PR as their individual PR.

 sources: add shimpei

 shimpei is an OCI shim wrapper around `oci-add-hooks`, whose sole
 porpuse is to call `oci-add-hooks` with additional parameters that can't
 be provided by containerd.
packages: add oci-add-hooks

oci-add-hooks is built as part of the release package, but it isn't included in any variant. Instead, packages that depend on it will include it in their spec file, as follows:

Requires: %{_cross_os}oci-add-hooks

Testing done:
In the new variant created in #1799:

I created a new containerd configuration that calls shimpei instead of runc:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia.options]
SystemdCgroup = true
BinaryName = "nvidia-oci" # this is a symlink to `shimpei`, used to figure out the `hooks.json` file 

The prestart hook in #1799, sets up GPUs and mounts helper binaries like nvidia-smi. In my test, I called nvidia-smi in a GPU-enabled orchestrated container:

kubectl exec -it nvidia-device-plugin-daemonset-4q2zf -n kube-system -- /bin/bash
root@nvidia-device-plugin-daemonset-4q2zf:/# nvidia-smi
Mon Dec 13 20:33:53 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: N/A      |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  Off  | 00000000:00:1E.0 Off |                    0 |
| N/A   29C    P0    24W / 300W |      0MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

packages/release/Cargo.toml Show resolved Hide resolved
sources/shimpei/src/main.rs Outdated Show resolved Hide resolved
sources/shimpei/src/main.rs Outdated Show resolved Hide resolved
@bcressey bcressey self-requested a review December 13, 2021 21:02
@bcressey
Copy link
Contributor

I expect this will build successfully once the files are uploaded to the lookaside cache.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
shimpei is an OCI shim wrapper around `oci-add-hooks`, whose sole
porpuse is to call `oci-add-hooks` with additional parameters that can't
be provided by containerd.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
@arnaldo2792
Copy link
Contributor Author

arnaldo2792 commented Dec 13, 2021

Forced push includes:

  • Add comment explaining why oci-add-hooks is in release/Cargo.toml
  • Nits in shimpei's messages

I also updated the lookaside cache

@arnaldo2792 arnaldo2792 merged commit 76ca753 into bottlerocket-os:develop Dec 14, 2021
@arnaldo2792 arnaldo2792 deleted the shimpei branch January 26, 2022 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants