-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Proposal] Introduce "sandboxer" plugin #7739
Comments
Based on sandbox api, I think this proposal would use a sandboxer to manage all pods, which can reduce much memory overhead of shim. |
(For others - there were some discussions on the PR too: #7502) Regarding shims, it would be better to start from the problems of containerd's shims rather than citing other shims such as dockershim in Kubernetes. They are different. |
We made a presensation about this proposal in the last meeting. and here is the slides: https://docs.google.com/presentation/d/1-6LXpWbQa7sw63KF1guDbHStcH4I9TgT/edit?usp=drivesdk&ouid=109476472776399967160&rtpof=true&sd=true For anyone who is interested. |
removed the citing of other shims |
@dmcgowan Could you help add a 2.0 milestone? |
|
What is the problem you're trying to solve
After the work of #4131, the "sandbox" becomes the first class citizen in containerd, and no more "pause container" disguising a sandbox, bravo!
Now the Sandbox Controller has two implementations:
Except for the pause container sandbox controller, the only option to implement the sandbox api is to integrate it to shim. which makes a "fat shim".
The new Sandbox API may be the ending of containerd shim. we can require that each sandbox should expose the Task API that containerd can call to manage container lifecycle. Then for the vm based sandbox, the agent in VM can expose Task API by virtio-vsock, so that containerd can call directly to the agent in vm to start a container, we don't need an extra API conversion from host to agent as kata is currently doing. This can reduce the memory overhead and API invoking path.
Describe the solution you'd like
We can use the powerfull plugin machanism to achieve it, Here we introduce a "Sandboxer" plugin to containerd. like "Snapshotter" to snapshot, a "Sandboxer" is for lifecycle and resource management of one kind of sandbox, we can provide different kinds of sandboxers, such as hypervisor based sandboxer, cgroup/ns based sandboxer, and wasm based sandboxer that may appear in the future.
The shim process is not required for sandboxes other than runc(we need a process to do tty io copy, if we don't need container that use tty, like all kubernetes launched containers do, we can start runc task server in containerd and remove the shim process, see #7502).
The Sandboxer can be either a TTRPC plugin running in containerd, or a remote plugin running outside the containerd, which is only one process on each node. containerd do not need to manage the shim process lifecycle anymore, because there is no shim process anymore.
Additional context
Like what Snapshotter did, containerd will collect all "Sandboxer"s when init
We may add three APIs to Sandbox Controller for update sandbox resources when start/stop/update a container:
Here are sequence diagrams when Sandboxer plugin introduced:
we have a demo PR #7502
The text was updated successfully, but these errors were encountered: