Repository navigation
Is it possible to speedup image unpacking on Windows? #14111
|
Currently, unpacking on Windows:
With big images (tens of GB), this becomes a serious problem even on fast SSDs because containerd is walking the whole image contents multiple times. Is it possible to replace steps 3+4 with just a single rename? Or, is it possible to unpack directly into I do understand this requires changes to containerd code, but just want to figure out if there is a conceptual reason why things are done the way they are done or this is just a lack of optimization. So, the problem is not image decompression (I already use zstd for my images), it is what happens after that, purely I/O-bound. |
Replies: 1 comment 1 reply
|
There's a conceptual reason, and it's worth noting up front that containerd isn't actually the component doing the staging — it hands the whole thing to hcsshim. The Windows differ just applies the tar straight at the snapshot path: // plugins/diff/windows/windows.go
archiveOpts := []archive.ApplyOpt{
archive.WithParents(parentLayerPaths),
archive.WithNoSameOwner(), // Lchown is not supported on Windows
archive.AsWindowsContainerLayer(),
}
if _, err := archive.Apply(ctx, layerPath, rc, archiveOpts...); err != nil {and // pkg/archive/tar_opts_windows.go
size, innerErr = ociwclayer.ImportLayerFromTar(ctx, r, root, options.Parents)So steps 2–4 in your list all live in Why it can't just be a renameThe temp directory doesn't contain the layer. It contains an intermediate transport format, and // internal/wclayer/importlayer.go
// ImportLayer will take the contents of the folder at importFolderPath and import
// that into a layer with the id layerId. Note that in order to correctly populate
// the layer and interperet the transport format, all parent layers must already
// be present on the system at the paths provided in parentLayerPaths.That "all parent layers must already be present" is the crux. The staged files are written through
A rename would move the backup-stream staging tree verbatim, and none of that resolution would happen. So steps 3+4 aren't a redundant copy of the finished layer — they're the conversion into it. What you can actually doBase layers already skip the temp entirely. In if len(parentLayerPaths) == 0 {
// This is a base layer. It gets imported differently.
f, err := safefile.OpenRoot(path)
...
return &baseLayerWriter{ctx: ctx, s: span, root: f}, nil
}
importPath, err := os.MkdirTemp("", "hcs")Only layers with parents pay the cost, which for a tens-of-GB image is likely most of them. You can at least move the staging. That's a plain The real answer upstream is CimFS. containerd already has |
There's a conceptual reason, and it's worth noting up front that containerd isn't actually the component doing the staging — it hands the whole thing to hcsshim.
The Windows differ just applies the tar straight at the snapshot path:
and
AsWindowsContainerLayerresolves to:So steps …