Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: garbage collection page #16032

Merged
merged 1 commit into from
Nov 1, 2022
Merged

Conversation

crazy-max
Copy link
Member

@crazy-max crazy-max commented Oct 31, 2022

needs #16036
related to docker/cli#2864 and moby/buildkit#1169

Was wondering if we should split this page and add some bits to dockerd reference page and BuildKit repo but I think it's fine to have it there as we already have all kind of information about caching. We could also cross link in a follow-up.

Also adds a new Cache section. Let me know if that sgty.

Signed-off-by: CrazyMax crazy-max@users.noreply.github.com

@netlify
Copy link

netlify bot commented Oct 31, 2022

Deploy Preview for docsdocker ready!

Name Link
🔨 Latest commit 1947bec
🔍 Latest deploy log https://app.netlify.com/sites/docsdocker/deploys/63613368868c860008c07de6
😎 Deploy Preview https://deploy-preview-16032--docsdocker.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@crazy-max crazy-max added the area/build Relates to Dockerfiles or docker build command label Oct 31, 2022
@crazy-max crazy-max marked this pull request as ready for review October 31, 2022 07:31
Copy link
Contributor

@jedevc jedevc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woo a page on buildkit gc 🎉

I wonder about page naming - maybe we could call it "Build cache garbage collection"? We have different types of gc in docker, with networks/volumes/containers/images and now build, so it might get confusing when searching for information?

I think it's good to have build cache gc information in the build cache section, but we should x-link it from wherever we talk about the other forms of cache (if that exists?).

Comment on lines +58 to +78
```
GC Policy rule#0:
All: false
Filters: type==source.local,type==exec.cachemount,type==source.git.checkout
Keep Duration: 48h0m0s
Keep Bytes: 512MB
GC Policy rule#1:
All: false
Keep Duration: 1440h0m0s
Keep Bytes: 26GB
GC Policy rule#2:
All: false
Keep Bytes: 26GB
GC Policy rule#3:
All: true
Keep Bytes: 26GB
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this come from, it looks like command output? (partly for curiosity, but also we should include the command used so that users can check their own policies themselves).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using buildctl to retrieve this info from a BuildKit container:

$ docker exec -it buildx_buildkit_builder20 buildctl debug workers -v
ID:             vhcmyn5zy0ub763bj4thm0l6c
Platforms:      linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6
BuildKit:       github.com/moby/buildkit v0.10.0-607-gf394afc59.m f394afc591a578537c3242a09c8b876db14a6d7f.m
Labels:
        org.mobyproject.buildkit.worker.executor:               oci
        org.mobyproject.buildkit.worker.hostname:               docker-desktop
        org.mobyproject.buildkit.worker.network:                host
        org.mobyproject.buildkit.worker.oci.process-mode:       sandbox
        org.mobyproject.buildkit.worker.snapshotter:            overlayfs
GC Policy rule#0:
        All:            false
        Filters:        type==source.local,type==exec.cachemount,type==source.git.checkout
        Keep Duration:  48h0m0s
        Keep Bytes:     512MB
GC Policy rule#1:
        All:            false
        Keep Duration:  1440h0m0s
        Keep Bytes:     26GB
GC Policy rule#2:
        All:            false
        Keep Bytes:     26GB
GC Policy rule#3:
        All:            true
        Keep Bytes:     26GB

Maybe we could add this in buildx when inspecting a builder? 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh awesome, ok, let's not put that command in then, since it's a bit fiddly 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could add this in buildx when inspecting a builder? 🤔

Yes! That would be really useful info to pull through 🎉

@crazy-max
Copy link
Member Author

crazy-max commented Oct 31, 2022

Moving to draft as it needs #16036 for cross links to BuildKit config.

@crazy-max crazy-max marked this pull request as draft October 31, 2022 15:30
@crazy-max crazy-max marked this pull request as ready for review October 31, 2022 15:41
@crazy-max crazy-max marked this pull request as draft October 31, 2022 15:41
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
@crazy-max crazy-max marked this pull request as ready for review November 1, 2022 14:55
Copy link
Collaborator

@dvdksn dvdksn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@crazy-max crazy-max merged commit a2f0cca into docker:main Nov 1, 2022
@crazy-max crazy-max deleted the build-cache-gc branch November 1, 2022 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build Relates to Dockerfiles or docker build command
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants