New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(go): add govender for the build process #3674

Merged
merged 4 commits into from Jul 23, 2017

Conversation

Projects
None yet
3 participants
@ellerbrock
Contributor

ellerbrock commented Jul 4, 2017

Docker Image Size Optimization

update for: #3666

size matters!

Cheers Maik

ellerbrock added some commits Jun 30, 2017

@anthonyfok anthonyfok self-requested a review Jul 5, 2017

@bep

This comment has been minimized.

Show comment
Hide comment
@bep

bep Jul 22, 2017

Member

@anthonyfok you self-requested a review ... any plans on doing that?

Member

bep commented Jul 22, 2017

@anthonyfok you self-requested a review ... any plans on doing that?

@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Jul 23, 2017

Contributor

@anthonyfok you self-requested a review ... any plans on doing that?

Yes, I am actually still waiting for @ellerbrock's reply to my review comment above, though I guess it is easy to miss on GitHub, so let me ask @ellerbrock this question again.


Hi @ellerbrock,

Could you please check and see if a $GOPATH/.cache directory exists?
Testing on Debian, govendor get caches all the dependencies it fetches inside the $GOPATH/.cache/govendor/ directory, and this directory totals 129MB here.

So, yes, would changing

  rm -rf pkg src bin/govendor && \

to

  rm -rf pkg src .cache bin/govendor && \

make the image smaller? :-)

Contributor

anthonyfok commented Jul 23, 2017

@anthonyfok you self-requested a review ... any plans on doing that?

Yes, I am actually still waiting for @ellerbrock's reply to my review comment above, though I guess it is easy to miss on GitHub, so let me ask @ellerbrock this question again.


Hi @ellerbrock,

Could you please check and see if a $GOPATH/.cache directory exists?
Testing on Debian, govendor get caches all the dependencies it fetches inside the $GOPATH/.cache/govendor/ directory, and this directory totals 129MB here.

So, yes, would changing

  rm -rf pkg src bin/govendor && \

to

  rm -rf pkg src .cache bin/govendor && \

make the image smaller? :-)

Show outdated Hide outdated Dockerfile
@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Jul 23, 2017

Contributor

@anthonyfok you self-requested a review ... any plans on doing that?

Oh! My bad! It turns out that I had a "pending" review which I wrote over two weeks ago, but one which only I myself could see.

I thought I had already "submitted" the review, and that the "pending" tag means "pending user feedback", but of course that was not the case.

Thanks for the heads up, @bep!

Contributor

anthonyfok commented Jul 23, 2017

@anthonyfok you self-requested a review ... any plans on doing that?

Oh! My bad! It turns out that I had a "pending" review which I wrote over two weeks ago, but one which only I myself could see.

I thought I had already "submitted" the review, and that the "pending" tag means "pending user feedback", but of course that was not the case.

Thanks for the heads up, @bep!

@ellerbrock

This comment has been minimized.

Show comment
Hide comment
@ellerbrock

ellerbrock Jul 23, 2017

Contributor

i quick added the .cache to the dockerfile.
won't hurt in case it would not be there :)

Contributor

ellerbrock commented Jul 23, 2017

i quick added the .cache to the dockerfile.
won't hurt in case it would not be there :)

@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Jul 23, 2017

Contributor

i quick added the .cache to the dockerfile.
won't hurt in case it would not be there :)

Thank you @ellerbrock! It does make a huge difference!

Here are the test results:

$ docker images 
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hugo-govendor-2     latest              37bbabca60b0        About an hour ago   277 MB
hugo-govendor-1     latest              eada10e25034        About an hour ago   400 MB
hugo-before-this-pr latest              b6fb765802dd        9 hours ago         428 MB

Looking at the actual layer that contains the /go/bin/hugo executable, the relevant layer.tar (created by docker image save shrinked from 140MB to 18MB:

Without rm -rf .cache (140MB)

$ tar tf /tmp/hugo-govendor-1/1d4e9012bde41481b33e7bc05986e43c2dbae3fa2989944a67a48d8f3570d20f/layer.tar
etc/
etc/apk/
etc/apk/world
etc/group
etc/group-
etc/passwd
etc/passwd-
etc/shadow
etc/shadow-
go/
go/.cache/
go/.cache/govendor/
go/.cache/govendor/github.com/
go/.cache/govendor/github.com/BurntSushi/
go/.cache/govendor/github.com/BurntSushi/toml/
go/.cache/govendor/github.com/BurntSushi/toml/.git/
go/.cache/govendor/github.com/BurntSushi/toml/.git/HEAD
go/.cache/govendor/github.com/BurntSushi/toml/.git/ORIG_HEAD
go/.cache/govendor/github.com/BurntSushi/toml/.git/branches/
go/.cache/govendor/github.com/BurntSushi/toml/.git/config
go/.cache/govendor/github.com/BurntSushi/toml/.git/description
go/.cache/govendor/github.com/BurntSushi/toml/.git/hooks/
go/.cache/govendor/github.com/BurntSushi/toml/.git/hooks/applypatch-msg.sample

... 5900+ lines omitted ...

go/.cache/govendor/gopkg.in/yaml.v2/resolve.go
go/.cache/govendor/gopkg.in/yaml.v2/scannerc.go
go/.cache/govendor/gopkg.in/yaml.v2/sorter.go
go/.cache/govendor/gopkg.in/yaml.v2/suite_test.go
go/.cache/govendor/gopkg.in/yaml.v2/writerc.go
go/.cache/govendor/gopkg.in/yaml.v2/yaml.go
go/.cache/govendor/gopkg.in/yaml.v2/yamlh.go
go/.cache/govendor/gopkg.in/yaml.v2/yamlprivateh.go
go/bin/
go/bin/hugo
go/.wh.src
lib/
lib/apk/
lib/apk/db/
lib/apk/db/installed
lib/apk/db/scripts.tar
lib/apk/db/triggers
site/
usr/
usr/bin/
usr/bin/dumb-init
var/
var/cache/
var/cache/misc/
var/cache/misc/busybox-1.26.2-r5.trigger

With rm -rf .cache (18MB)

$ tar tf /tmp/hugo-govendor-2/0e3353507920dd0e1db6f794c21c57faa867df2159fb96b3f19c79ae9f49099e/layer.tar 
etc/
etc/apk/
etc/apk/world
etc/group
etc/group-
etc/passwd
etc/passwd-
etc/shadow
etc/shadow-
go/
go/bin/
go/bin/hugo
go/.wh.src
lib/
lib/apk/
lib/apk/db/
lib/apk/db/installed
lib/apk/db/scripts.tar
lib/apk/db/triggers
site/
usr/
usr/bin/
usr/bin/dumb-init
var/
var/cache/
var/cache/misc/
var/cache/misc/busybox-1.26.2-r5.trigger
Contributor

anthonyfok commented Jul 23, 2017

i quick added the .cache to the dockerfile.
won't hurt in case it would not be there :)

Thank you @ellerbrock! It does make a huge difference!

Here are the test results:

$ docker images 
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hugo-govendor-2     latest              37bbabca60b0        About an hour ago   277 MB
hugo-govendor-1     latest              eada10e25034        About an hour ago   400 MB
hugo-before-this-pr latest              b6fb765802dd        9 hours ago         428 MB

Looking at the actual layer that contains the /go/bin/hugo executable, the relevant layer.tar (created by docker image save shrinked from 140MB to 18MB:

Without rm -rf .cache (140MB)

$ tar tf /tmp/hugo-govendor-1/1d4e9012bde41481b33e7bc05986e43c2dbae3fa2989944a67a48d8f3570d20f/layer.tar
etc/
etc/apk/
etc/apk/world
etc/group
etc/group-
etc/passwd
etc/passwd-
etc/shadow
etc/shadow-
go/
go/.cache/
go/.cache/govendor/
go/.cache/govendor/github.com/
go/.cache/govendor/github.com/BurntSushi/
go/.cache/govendor/github.com/BurntSushi/toml/
go/.cache/govendor/github.com/BurntSushi/toml/.git/
go/.cache/govendor/github.com/BurntSushi/toml/.git/HEAD
go/.cache/govendor/github.com/BurntSushi/toml/.git/ORIG_HEAD
go/.cache/govendor/github.com/BurntSushi/toml/.git/branches/
go/.cache/govendor/github.com/BurntSushi/toml/.git/config
go/.cache/govendor/github.com/BurntSushi/toml/.git/description
go/.cache/govendor/github.com/BurntSushi/toml/.git/hooks/
go/.cache/govendor/github.com/BurntSushi/toml/.git/hooks/applypatch-msg.sample

... 5900+ lines omitted ...

go/.cache/govendor/gopkg.in/yaml.v2/resolve.go
go/.cache/govendor/gopkg.in/yaml.v2/scannerc.go
go/.cache/govendor/gopkg.in/yaml.v2/sorter.go
go/.cache/govendor/gopkg.in/yaml.v2/suite_test.go
go/.cache/govendor/gopkg.in/yaml.v2/writerc.go
go/.cache/govendor/gopkg.in/yaml.v2/yaml.go
go/.cache/govendor/gopkg.in/yaml.v2/yamlh.go
go/.cache/govendor/gopkg.in/yaml.v2/yamlprivateh.go
go/bin/
go/bin/hugo
go/.wh.src
lib/
lib/apk/
lib/apk/db/
lib/apk/db/installed
lib/apk/db/scripts.tar
lib/apk/db/triggers
site/
usr/
usr/bin/
usr/bin/dumb-init
var/
var/cache/
var/cache/misc/
var/cache/misc/busybox-1.26.2-r5.trigger

With rm -rf .cache (18MB)

$ tar tf /tmp/hugo-govendor-2/0e3353507920dd0e1db6f794c21c57faa867df2159fb96b3f19c79ae9f49099e/layer.tar 
etc/
etc/apk/
etc/apk/world
etc/group
etc/group-
etc/passwd
etc/passwd-
etc/shadow
etc/shadow-
go/
go/bin/
go/bin/hugo
go/.wh.src
lib/
lib/apk/
lib/apk/db/
lib/apk/db/installed
lib/apk/db/scripts.tar
lib/apk/db/triggers
site/
usr/
usr/bin/
usr/bin/dumb-init
var/
var/cache/
var/cache/misc/
var/cache/misc/busybox-1.26.2-r5.trigger

@anthonyfok anthonyfok merged commit 606d6a8 into gohugoio:master Jul 23, 2017

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla Contributor License Agreement is signed.
Details
@ellerbrock

This comment has been minimized.

Show comment
Hide comment
@ellerbrock

ellerbrock Jul 23, 2017

Contributor

nice optimization!
lets see how small we can get via #3730 ...

Contributor

ellerbrock commented Jul 23, 2017

nice optimization!
lets see how small we can get via #3730 ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment