-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for exporting nydus compression type #2581
Commits on Nov 1, 2022
-
cache: support nydus compression type
Nydus image is a container accelerated image format provided by the Dragonfly image-service project, which offers the ability to pull image data on demand, without waiting for the entire image pull to complete and then start the container. It has been put in production usage and shown vast improvements over the old OCI image format in terms of container launching speed, image space, and network bandwidth efficiency, as well as data integrity. Nydus image can be flexibly configured as a FUSE-based user-space filesystem or in-kernel EROFS (from Linux kernel v5.16) with Nydus daemon in user-space, integrating with VM-based container runtime like KataContainers is much easier. Nydus has provided a conversion tool Nydusify for converting OCIv1 image to Nydus image and integrated into Harbor Acceld as a conversion driver, which assumes that the OCI image is already available in the registry, but a better way would be to build the Nydus images directly from the build system instead of using the conversion tool, which would increase the speed of the image export, so we experimentally integrated the Nydus export in Buildkit. Unlike other compression formats (gzip, estargz, etc.) in OCI image, nydus is divided into two types of layer, blob, and bootstrap, where blob serves as the data part of each layer of the image, and bootstrap serves as the metadata of the whole image, the bootstrap is equivalent to the view of the whole image filesystem after all layers overlay. For example, for an OCI image with 3 layers, the corresponding nydus image is 4 layers (3 layers of blob + 1 layer of bootstrap). The nydus-snapshotter project provides a package to do the actual layer compression, this commit imports the package to implement the export of nydus compression type. Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
Configuration menu - View commit details
-
Copy full SHA for cf70ab6 - Browse repository at this point
Copy the full SHA cf70ab6View commit details -
exporter: append extra layer for nydus compression type
For nydus image, appending an extra nydus bootstrap layer to the manifest, this layer represents the whole metadata of filesystem view for the entire image. Nydus uses this extra boostrap layer to reduce the number of HTTP request and FUSE overlay's overhead on preparing the container's rootfs. For now we just support this in image type exporter. Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
Configuration menu - View commit details
-
Copy full SHA for 0d1f8c4 - Browse repository at this point
Copy the full SHA 0d1f8c4View commit details -
test: add test for nydus compression type
Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
Configuration menu - View commit details
-
Copy full SHA for fa43b9f - Browse repository at this point
Copy the full SHA fa43b9fView commit details -
vendor: sync for supporting nydus compression type
Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
Configuration menu - View commit details
-
Copy full SHA for f53eb59 - Browse repository at this point
Copy the full SHA f53eb59View commit details -
doc: add usage for nydus compression type
Introduce nydus image and show that how to export nydus image, and add known limitations. Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
Configuration menu - View commit details
-
Copy full SHA for ab6e8bb - Browse repository at this point
Copy the full SHA ab6e8bbView commit details
Commits on Nov 3, 2022
-
nydus: adjust code structure for independence
Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
Configuration menu - View commit details
-
Copy full SHA for 3e9e898 - Browse repository at this point
Copy the full SHA 3e9e898View commit details