build(go): add govender for the build process

[ellerbrock]

Docker Image Size Optimization

• add govendor to build process of hugo

update for: #3666

Cheers Maik

[bep]

@anthonyfok you self-requested a review ... any plans on doing that?

[anthonyfok]

@anthonyfok you self-requested a review ... any plans on doing that?

Yes, I am actually still waiting for @ellerbrock's reply to my review comment above, though I guess it is easy to miss on GitHub, so let me ask @ellerbrock this question again.

---

Hi @ellerbrock,

Could you please check and see if a $GOPATH/.cache directory exists?
Testing on Debian, govendor get caches all the dependencies it fetches inside the $GOPATH/.cache/govendor/ directory, and this directory totals 129MB here.

So, yes, would changing

  rm -rf pkg src bin/govendor && \

to

  rm -rf pkg src .cache bin/govendor && \

make the image smaller? :-)

[anthonyfok]

Hi @ellerbrock,

Could you please check and see if a $GOPATH/.cache directory exists?
Testing on Debian, govendor get caches all the dependencies it fetches inside the $GOPATH/.cache/govendor/ directory, and this directory totals 129MB here.

So, yes, would changing

  rm -rf pkg src bin/govendor && \

to

  rm -rf pkg src .cache bin/govendor && \

make the image smaller? :-)

[ellerbrock]

hi @anthonyfok ,

thanks for the answers, 129MB would be a big optimization.
i had a look inside the container (https://hub.docker.com/r/ellerbrock/alpine-hugo/) but could not see any .cache folder.
Are there any new changes causing this or do i miss something?

docker run -it --entrypoint ash ellerbrock/alpine-hugo

/ $ cd /go
/go $ ls -alF
total 12
drwxrwxrwx    1 root     root          4096 Jul 21 16:39 ./
drwxr-xr-x    1 root     root          4096 Jul 23 06:20 ../
drwxrwxrwx    1 root     root          4096 Jul 21 16:39 bin/
/go $

Cheers Maik

[anthonyfok]

Hi @ellerbrock,

Thank you for your quick response.

The $GOPATH/.cache directory is created by govendor, though I can see that, unlike this pull request, you are not using actually govendor at https://hub.docker.com/r/ellerbrock/alpine-hugo/~/dockerfile/, hence no $GOPATH/.cache. 😉 (which personally I won't judge because, putting my Debian Developer hat on, both Debian and Fedora's Go packaging policy demands that we un-vendor everything, so govendor isn't used in packaging Hugo, but anyhow...)

I am now attempting to run sudo docker build to actually test your Dockerfile. (My hard disk is quite full; hope I won't run out of space doing so, haha.)

[anthonyfok]

@anthonyfok you self-requested a review ... any plans on doing that?

Oh! My bad! It turns out that I had a "pending" review which I wrote over two weeks ago, but one which only I myself could see.

I thought I had already "submitted" the review, and that the "pending" tag means "pending user feedback", but of course that was not the case.

Thanks for the heads up, @bep!

[ellerbrock]

i quick added the .cache to the dockerfile.
won't hurt in case it would not be there :)

[anthonyfok]

i quick added the .cache to the dockerfile.
won't hurt in case it would not be there :)

Thank you @ellerbrock! It does make a huge difference!

Here are the test results:

$ docker images 
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hugo-govendor-2     latest              37bbabca60b0        About an hour ago   277 MB
hugo-govendor-1     latest              eada10e25034        About an hour ago   400 MB
hugo-before-this-pr latest              b6fb765802dd        9 hours ago         428 MB

Looking at the actual layer that contains the /go/bin/hugo executable, the relevant layer.tar (created by docker image save shrinked from 140MB to 18MB:

Without rm -rf .cache (140MB)

$ tar tf /tmp/hugo-govendor-1/1d4e9012bde41481b33e7bc05986e43c2dbae3fa2989944a67a48d8f3570d20f/layer.tar
etc/
etc/apk/
etc/apk/world
etc/group
etc/group-
etc/passwd
etc/passwd-
etc/shadow
etc/shadow-
go/
go/.cache/
go/.cache/govendor/
go/.cache/govendor/github.com/
go/.cache/govendor/github.com/BurntSushi/
go/.cache/govendor/github.com/BurntSushi/toml/
go/.cache/govendor/github.com/BurntSushi/toml/.git/
go/.cache/govendor/github.com/BurntSushi/toml/.git/HEAD
go/.cache/govendor/github.com/BurntSushi/toml/.git/ORIG_HEAD
go/.cache/govendor/github.com/BurntSushi/toml/.git/branches/
go/.cache/govendor/github.com/BurntSushi/toml/.git/config
go/.cache/govendor/github.com/BurntSushi/toml/.git/description
go/.cache/govendor/github.com/BurntSushi/toml/.git/hooks/
go/.cache/govendor/github.com/BurntSushi/toml/.git/hooks/applypatch-msg.sample

... 5900+ lines omitted ...

go/.cache/govendor/gopkg.in/yaml.v2/resolve.go
go/.cache/govendor/gopkg.in/yaml.v2/scannerc.go
go/.cache/govendor/gopkg.in/yaml.v2/sorter.go
go/.cache/govendor/gopkg.in/yaml.v2/suite_test.go
go/.cache/govendor/gopkg.in/yaml.v2/writerc.go
go/.cache/govendor/gopkg.in/yaml.v2/yaml.go
go/.cache/govendor/gopkg.in/yaml.v2/yamlh.go
go/.cache/govendor/gopkg.in/yaml.v2/yamlprivateh.go
go/bin/
go/bin/hugo
go/.wh.src
lib/
lib/apk/
lib/apk/db/
lib/apk/db/installed
lib/apk/db/scripts.tar
lib/apk/db/triggers
site/
usr/
usr/bin/
usr/bin/dumb-init
var/
var/cache/
var/cache/misc/
var/cache/misc/busybox-1.26.2-r5.trigger

With rm -rf .cache (18MB)

$ tar tf /tmp/hugo-govendor-2/0e3353507920dd0e1db6f794c21c57faa867df2159fb96b3f19c79ae9f49099e/layer.tar 
etc/
etc/apk/
etc/apk/world
etc/group
etc/group-
etc/passwd
etc/passwd-
etc/shadow
etc/shadow-
go/
go/bin/
go/bin/hugo
go/.wh.src
lib/
lib/apk/
lib/apk/db/
lib/apk/db/installed
lib/apk/db/scripts.tar
lib/apk/db/triggers
site/
usr/
usr/bin/
usr/bin/dumb-init
var/
var/cache/
var/cache/misc/
var/cache/misc/busybox-1.26.2-r5.trigger

[ellerbrock]

nice optimization!
lets see how small we can get via #3730 ...

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.