Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dockertools buildImage: support new-style image specs #51528

Merged
merged 1 commit into from Dec 7, 2018

Conversation

@grahamc
Copy link
Member

grahamc commented Dec 4, 2018

Docker images used to be, essentially, a linked list of layers. Each
layer would have a tarball and a json document pointing to its parent,
and the image pointed to the top layer:

imageA  ----> layerA
                |
                v
              layerB
                |
                v
              layerC

The current image spec changed this format to where the Image defined
the order and set of layers:

imageA  ---> layerA
        |--> layerB
        `--> layerC

For backwards compatibility, docker produces images which follow both
specs: layers point to parents, and images also point to the entire
list:

imageA  ---> layerA
        |      |
        |      v
        |--> layerB
        |      |
        |      v
        `--> layerC

This is nice for tooling which supported the older version and never
updated to support the newer format.

Our buildImage code only supported the old version, so in order for
buildImage to properly generate an image based on another image
with fromImage, the parent image's layers must fully support the old
mechanism.

This is not a problem in general, but is a problem with
buildLayeredImage.

buildLayeredImage creates images with newer image spec, because
individual store paths don't have a guaranteed parent layer. Including
a specific parent ID in the layer's json makes the output less likely
to cache hit when published or pulled.

This means until now, buildLayeredImage could not be the input to
buildImage.

The changes in this PR change buildImage to only use the layer's
manifest when locating parent IDs. This does break buildImage on
extremely old Docker images, though I do wonder how many of these
exist.

This work has been sponsored by Target.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@grahamc grahamc requested review from globin and nlewo Dec 4, 2018
@grahamc

This comment has been minimized.

Copy link
Member Author

grahamc commented Dec 4, 2018

This is a "RFC" stage PR, I think there is a lot of opportunity for improvement here.

@nlewo
nlewo approved these changes Dec 5, 2018
Copy link
Member

nlewo left a comment

LGTM.

FYI, even for "layered" images, I think (not entirely sure) you could generate the old style format. I think in your description, your are forgotten an indirection which is the layer directory: a identical layer could exists in different "layer directory", each of these "layer directories" having its own local configuration. This layer configuration would be created at image creation time, when layers order is known.
But this is not really important since this format is deprecated:/

@nlewo

This comment has been minimized.

Copy link
Member

nlewo commented Dec 5, 2018

@GrahamcOfBorg test docker-tools

@grahamc

This comment has been minimized.

Copy link
Member Author

grahamc commented Dec 5, 2018

@GrahamcOfBorg build nixosTests.docker-tools

@grahamc

This comment has been minimized.

Copy link
Member Author

grahamc commented Dec 5, 2018

I wonder what sort of fundamental tooling we'd need for these scripts. Maybe we have a few functions here, hiding behind ugly bash:

  • given a directory + a bit of metadat, produce a layer
  • take a list of layers to produce an image
  • iterate over an image's layers

I took a look at umoci (https://umo.ci) and it seems interesting, though can't produce an individual layer. I'm afraid our unique approach to packages makes us require unique tooling ... I also looked at buildah https://github.com/containers/buildah but it seems to require capabilities which aren't available in a build sandbox.

Docker images used to be, essentially, a linked list of layers. Each
layer would have a tarball and a json document pointing to its parent,
and the image pointed to the top layer:

    imageA  ----> layerA
                    |
                    v
                  layerB
                    |
                    v
                  layerC

The current image spec changed this format to where the Image defined
the order and set of layers:

    imageA  ---> layerA
            |--> layerB
            `--> layerC

For backwards compatibility, docker produces images which follow both
specs: layers point to parents, and images also point to the entire
list:

    imageA  ---> layerA
            |      |
            |      v
            |--> layerB
            |      |
            |      v
            `--> layerC

This is nice for tooling which supported the older version and never
updated to support the newer format.

Our `buildImage` code only supported the old version, so in order for
`buildImage` to properly generate an image based on another image
with `fromImage`, the parent image's layers must fully support the old
mechanism.

This is not a problem in general, but is a problem with
`buildLayeredImage`.

`buildLayeredImage` creates images with newer image spec, because
individual store paths don't have a guaranteed parent layer. Including
a specific parent ID in the layer's json makes the output less likely
to cache hit when published or pulled.

This means until now, `buildLayeredImage` could not be the input to
`buildImage`.

The changes in this PR change `buildImage` to only use the layer's
manifest when locating parent IDs. This does break buildImage on
extremely old Docker images, though I do wonder how many of these
exist.

This work has been sponsored by Target.
@grahamc grahamc force-pushed the grahamc:buildImage-on-layered-image branch to c88337c Dec 5, 2018
@grahamc

This comment has been minimized.

Copy link
Member Author

grahamc commented Dec 5, 2018

My force-push is identical to the version reviewed by @nlewo already, just squashed and with a nicer commit message.

@grahamc grahamc mentioned this pull request Dec 5, 2018
6 of 10 tasks complete
@nlewo

This comment has been minimized.

Copy link
Member

nlewo commented Dec 7, 2018

@grahamc hmm, yes, maybe we need our own tooling :(

We would also need a function that pushes an image on top of a parent image. In our current tooling, this part is quite complicated (to ignore already existing files). I wonder how this is required thanks to buildLayeredImage: all files are in the store and only links are created, so by construction, we avoid (at least reduce) the duplicated files problem.

Regarding the tooling, I wonder how a combination of umoci + skopeo could do the job:

  • createLayer :: Dir -> Layer
    umoci creates a single layered image and skopeo extracts its layer (skopeo copy dir://).
  • makeImage :: [Layer] -> Image
    we create a directory of layers and use skopeo copy dir://$dir docker-archive://$out to pack the image

But maybe it would be more complicated than creating our own tools :)

Another point is regarding the way we pack the image. With recursiveNix, I think we would like to dynamically create the image in order to not have to duplicate layers in the Nix store. In this case, the image format in the Nix store could be the skopeo dir where each layer would be a link to the layer in the Nix store.

@nlewo

This comment has been minimized.

Copy link
Member

nlewo commented Dec 7, 2018

Thx!

@nlewo nlewo merged commit f7e67be into NixOS:master Dec 7, 2018
12 checks passed
12 checks passed
nix-build -A nixosTests.docker-tools --argstr system x86_64-linux Build Results
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
nix-build -A nixosTests.docker-tools --argstr system aarch64-linux Build Results
Details
nix-build -A nixosTests.docker-tools --argstr system x86_64-darwin Build Results
Details
@grahamc grahamc deleted the grahamc:buildImage-on-layered-image branch Dec 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.