Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
dockertools buildImage: support new-style image specs #51528
Docker images used to be, essentially, a linked list of layers. Each
The current image spec changed this format to where the Image defined
For backwards compatibility, docker produces images which follow both
This is nice for tooling which supported the older version and never
This is not a problem in general, but is a problem with
This means until now,
The changes in this PR change
This work has been sponsored by Target.
nlewo left a comment
FYI, even for "layered" images, I think (not entirely sure) you could generate the old style format. I think in your description, your are forgotten an indirection which is the layer directory: a identical layer could exists in different "layer directory", each of these "layer directories" having its own local configuration. This layer configuration would be created at image creation time, when layers order is known.
I wonder what sort of fundamental tooling we'd need for these scripts. Maybe we have a few functions here, hiding behind ugly bash:
I took a look at umoci (https://umo.ci) and it seems interesting, though can't produce an individual layer. I'm afraid our unique approach to packages makes us require unique tooling ... I also looked at buildah https://github.com/containers/buildah but it seems to require capabilities which aren't available in a build sandbox.
Docker images used to be, essentially, a linked list of layers. Each layer would have a tarball and a json document pointing to its parent, and the image pointed to the top layer: imageA ----> layerA | v layerB | v layerC The current image spec changed this format to where the Image defined the order and set of layers: imageA ---> layerA |--> layerB `--> layerC For backwards compatibility, docker produces images which follow both specs: layers point to parents, and images also point to the entire list: imageA ---> layerA | | | v |--> layerB | | | v `--> layerC This is nice for tooling which supported the older version and never updated to support the newer format. Our `buildImage` code only supported the old version, so in order for `buildImage` to properly generate an image based on another image with `fromImage`, the parent image's layers must fully support the old mechanism. This is not a problem in general, but is a problem with `buildLayeredImage`. `buildLayeredImage` creates images with newer image spec, because individual store paths don't have a guaranteed parent layer. Including a specific parent ID in the layer's json makes the output less likely to cache hit when published or pulled. This means until now, `buildLayeredImage` could not be the input to `buildImage`. The changes in this PR change `buildImage` to only use the layer's manifest when locating parent IDs. This does break buildImage on extremely old Docker images, though I do wonder how many of these exist. This work has been sponsored by Target.
@grahamc hmm, yes, maybe we need our own tooling :(
We would also need a function that pushes an image on top of a parent image. In our current tooling, this part is quite complicated (to ignore already existing files). I wonder how this is required thanks to
Regarding the tooling, I wonder how a combination of
But maybe it would be more complicated than creating our own tools :)
Another point is regarding the way we pack the image. With