Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: optionally export cache metadata in the image config #752

Closed
tonistiigi opened this issue Dec 11, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@tonistiigi
Copy link
Member

commented Dec 11, 2018

BuildKit supports exporting build cache and pulling back the parts that it knows will produce matches. There are currently 2 export modes: max that will export all the layers of all intermediate steps and min that will only export layers for the resulting image(that are presumably exported anyway) and only metadata for all the intermediate steps. Having only metadata for the intermediate steps is already very beneficial in lots of cases and can still effectively cache multi-stage builds.

Currently, the UI for exporting cache in docker does not exist and because the cache format uses manifest lists and docker's buildkit integration is not using containerd it is unlikely that it can be added for 19.03.

As an alternative, we could support sharing cache inside the image config that we should be able to easily use in docker as well.

Basically, we would do the same thing we do with current min cache mode but instead of pushing the metadata+final layers to a separate reference with a manifest list we would inline the metadata directly into image config. The image config already has the layer information.

{
   ....
   "moby.buildkit.buildcache": <base64>
}

Then this image can be used directly in the docker build --cache-from.

Note that one of the problems with the implementation is that while currently, the image config(digest) is reproducible in case of a cache match, the build cache atm is not. This is because the local sources use random IDs for their definition based cache keys. So either this issue needs to be solved and these cache keys removed from the exported selection (eg. marking these keys as not for exporting) or we need to accept new digests on every build(old docker behavior).

This should also be optional and cache should only be produced when it is requested.

@AkihiroSuda

This comment has been minimized.

Copy link
Member

commented Dec 20, 2018

SGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.