Add multi OS support #439
Comments
|
A few thoughts: DefinitionsWe should specify what we mean by "path" (I vote for only accepting absolutes). BehaviourI think we should document the expected behaviour if either of those annotations are blank or invalid in other ways (immediate error or fallback to defaults with a big fat log warning message :) Kernel/Image selectionAs mentioned on the original OCI PR, we need a way to select between images and kernels. Currently for CC 3.0, the runtime handles that via its configuration file. These annotations suggest a broadening of the ability to switch between images / kernels by allowing the caller (the user running, say,
If it's the runtime, I wonder if we need to find a way for workloads to be tagged as needing particular features (such as NFV) to allow the runtime to decide automatically which kernel to choose? That might require we mandate another annotation for SecurityWe could add checksum annotations for the kernel + image, but that would significantly slow down startup time (if they were being checked). Hence, we should probably document that other facilities (such as package-management features) are used to ensure the images have not been modified. Stale component versionsWhatever we decide, we need to ensure the runtime is able to identify stale containers. If the kernel+image selection is not going to be decided by the runtime, the runtime will still need a way to identify the range of available images+kernels, and be able to order them by age to be able to identify "stale" containers. |
|
@jodh-intel Thanks for the feedback. Some comments:
I'd like to look at the problem only from a virtcontainers perspective for now, and I think virtcontainers should not try to be smart about it, but simply work on kernel and/or image paths. Then runtime implementations can decide if they want to resolve higherl level requests into actual paths or not.
That would be an interesting feature, that we could decide to disable by default unless the caller explicitly asks for it. And this could also apply to the default kernel + image files. I'll add that to the issue.
With this new approach we will still be able to have
|
|
@sameo @jodh-intel this feature change should be accompanied by the ability to retrieve the kernel and runtime in use for containers/pods on a given system using the cc-runtime binary. This may be the same as what we are planning to do to report the version of the kernel and image in use. Another thing we may want to consider is allowing module loading in clear containers. If we allow for module loading, we will be able to support significantly more combinations and allow the workload to be self contained, and the the workload itself can carry the modules it needs. The module loading support does not need to be supported in the default kernel. Also we should explore if node tagging can be easily extended to report the availability of various kernels/rootfs combinations. Today there is no hard linkage between the kernel and rootfs, as they are loosely coupled. But keeping this loose coupling may be an issue in the future, when we allow the kernels to change. |
With this new approach we will still be able to know which kernel+image each pod is using by calling
Agreed. This will have little impact on us, except maybe that we will need to add
So here what we'd most likely want to know is which kernels are compatible with any given image, I think. Just a thought: We could have osbuilder generate a metadata file for any generated image that would point to the compatible kernel(s) for it. |
|
If we do make a module enabled kernel then I'd recommend ensuring we have module versioning turned on to help ensure we only load modules that were build for that actual kernel: We don't want to be tracking down issues where somebody has loaded an older or newer module which causes 'strange things to happen'. |
|
cc @eadamsintel |
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. If we find a valid path for a kernel or an image, we change the hypervisor config to point to them. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. If we find a valid path for a kernel or an image, we change the hypervisor config to point to them. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We add an annotation package with all virtcontainers public annotations. For now we will add the assets paths and hashes annotations. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Based on the pod config annotations, we change the underlying hypervisor configuration. Pod config annotations can carry a kernel path, a guest image path or both. Those annotations, if valid, will overwrite the hypervisor configuration paths. If not valid, virtcontainers will error out. Optionally, asset SHA-512 hashes can be passed through pod annotations as well. In that case virtcontainers will verify that the asset binary matches the hash annotation and error out when it does not. Fixes #439 Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Problem statement
One key advantage of Clear Containers over
runccontainers is the ability for each container/pod to run on top of its own kernel and guest image.For customized workloads like e.g. NFV ones, this is very important as they sometimes need some specific kernel features that can be mutually exclusive with other workload needs.
Unfortunately, the OCI runtime specifications do not allow (yet) for specifying a kernel and guest image and thus
virtcontainerscallers can only use a default path for each of those and pass it through theHypervisorConfigstructure.Proposed solutions
We want to be able to define a kernel and a guest image paths per container/pod. Both of them should be absolute paths, and should be optional. In other words, there must be a default value for those paths which will be passed to virtcontainers through the
HypervisorConfigstructure. Clear Containers, for example, gets the default paths for its kernel and guest image through theconfiguration.tomlsystem wide configuration file.Short term solution: Annotations
As a short term solution we are going to use virtcontainers namespaced pod annotations for passing a kernel and guest image absolute path:
Both annotations can be set through
virtcontainersPodConfigannotations. Kernel or image pathContainerConfigannotations will be ignored.Long term solution: OCI specification changes
See opencontainers/runtime-spec#405
Code path
The virtcontainers code path should be identical for both solutions: The
pod.gowill trap pod creation request and modify itsconfig.HypervisorConfigstructure if there is anImagePathand/orKernelPathannotation as part of itsPodConfigThe
pkg/oci/utils.gofile will be responsible for adding the right pod annotation, either by parsing the OCIconfig.jsonannotations or by parsing the future VM specific section in that file. The latter should take precedence over the former.Error handling
If any of the kernel or guest image path is invalid (empty files, or inexistent ones),
virtcontainerswill error out and stop creating the pod's VM. Passing an image or/and a guest image into the container configuration file is a very specific requirement from the user and thus switching to the default value when one of those paths is invalid could be misleading and error prone.Security
Optionally
virtcontainerscallers can require kernel and/or image binaries integrity checking. For that purpose,virtcontainerswill handle 2 additional annotations:By default
virtcontainerswill not verify the kernel and image binaries integrity unless one or both of the hash annotations are passed in thePodConfigannotations. In the latter case,virtcontainerswill build SHA-512 hashes on the kernel or/and image binaries and compare that against the passed annotations.The text was updated successfully, but these errors were encountered: