Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: early stage layers from multi-stage builds should not be considered dangling #38071

Open
daveisfera opened this Issue Oct 23, 2018 · 7 comments

Comments

Projects
None yet
5 participants
@daveisfera
Copy link

daveisfera commented Oct 23, 2018

This is a bit of the opposite request from #34151 and #34513 because currently we have to explicitly tag the early stages so that docker image prune doesn't clean them up and blow away build cache that should still be valid.

The current workaround is to run the following:

docker build --target build -t myimage_build myimage
docker build -t myimage myimage

This is inefficient because the build for the build stage has to happen twice. The second time it's all cached, but it means that the context has to be reloaded and this explicit tagging is required when it shouldn't be necessary because the image isn't dangling and is part of a tagged build.

@cpuguy83

This comment has been minimized.

Copy link
Contributor

cpuguy83 commented Oct 24, 2018

I think with buildkit enabled this may not be an issue, but I'm not positive.
Have you tried, with Docker 18.06 in experimental mode, with DOCKER_BUILDKIT=1 docker build <args>?

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Oct 24, 2018

Yes, with buildkit, the build-cache no longer creates those images; build-cache is kept separate from the image store, and can be pruned separately (docker build prune; added through docker/cli#1295 in docker 18.09).

Docker 18.09 also brings a configurable garbage-collect policy for the build-cache (see #37846 for details)

@daveisfera

This comment has been minimized.

Copy link
Author

daveisfera commented Oct 24, 2018

Is there documentation on this functionality somewhere? I'd like to understand how it works and make sure that it will meet out needs.

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Oct 24, 2018

Most information right now can be found in those pull requests (more documentation still has to be written), but you can try a release candidate of 18.09 to test those features, and check if the behavior matches your expectations / use-case.

@daveisfera

This comment has been minimized.

Copy link
Author

daveisfera commented Oct 24, 2018

Is there a way to view the build-cache?

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Oct 24, 2018

The files themselves? Not sure tbh (@tiborvass @tonistiigi @AkihiroSuda should know). If I'm not mistaken, the size will show up in docker system df

@tiborvass

This comment has been minimized.

Copy link
Collaborator

tiborvass commented Oct 24, 2018

@daveisfera Build cache can be viewed with docker system df -v, and in master docker cli (or 18.09) there is a way to get --format {{json .}} in conjunction with -v, so you could do docker system df -v --format {{json .}} | jq .BuildCache

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.