Reduce Docker image size further using "multi-stage builds" in Docker CE 17.06

[anthonyfok]

Thanks to @ellerbrock's optimization effort in #3666 and #3674, the Hugo Docker image generated using the Dockerfile was greatly reduced from 428MB to 277MB, but it still looks a bit big.

It turns out that about 250MB is taken up by /usr/local/go, which is needed for building Hugo, but not needed for running Hugo. So, that got me wondering: Could we build the Hugo Docker image with golang:alpine3.6 (258 MB) like we do now, but use the much smaller alpine:latest (3.96 MB) for the running image?

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
golang              alpine3.6           9f687d7d2a96        3 weeks ago         258 MB
alpine              latest              7328f6f8b418        3 weeks ago         3.96 MB

That way, perhaps future Hugo Docker images could be smaller than 25 MB!

It turns out that it was not a trivial thing to do before, but with the hot new Docker CE 17.06 (released 2017-06-28) and its "Multi-stage Builds" feature, it looks like this goal is within reach:

• Even Smaller Docker Image Sizes
• Announcing Docker 17.06 Community Edition (CE) - Docker Blog

@ellerbrock, could you please look into this when you are free? (No hurries though, as I am sure it will take a while for Docker CE 17.06 to propagate, e.g. Debian is still at Docker 1.13.1 released in February 2017.) Many thanks!

[ellerbrock]

Hi @anthonyfok, good idea.
There are a few ways how we could archive this till the new feature is official supported in Docker:

1.) In most cases i currently build and cleanup inside the container. In that case its isolated without any further outside dependencies. The important part here is to install and cleanup in one RUN command to avoid hitting the cache layer to keep the container size small.

2.) Another solution is to setup a proper CI pipeline (https://docs.travis-ci.com/user/languages/go/) and just copy the executable inside the container. I would only use such a solution if all steps are transparent to ensure that no malicious stuff finds a way inside the container and my network (with Travis which you use anyway this is possible).

3.) We could also setup a Automatic Build Task on Docker Hub and define a branch which will trigger a new build with each code push. This also keep it transparent. You can find examples on my Docker Hub Account @ellerbrock, all my containers are build this way. I could help with the setup an and automation it if you like. Would be nice anyway to have an official Hugo Docker Container available.

Let me know which of those solutions you and your team would prefer to use ...

[Update] Here is an interesting article which describes how to start FROM scratch and just add the statically linked go binary file.

Cheers Maik

[anthonyfok]

There are a few ways how we could archive this till the new feature is official supported in Docker:

Thanks @ellerbrock! I'd say we just wait until Multi-Stage Builds official lands in Docker Hub, which should happen really soon now, I guess some time this August? See docker/hub-feedback#1039

We are in no real hurry to shrink the Docker image size to < 25MB, so let's just take our time, wait, and do it properly once and for all when it is supported officially in Docker Hub infrastructure where we can use a very clean and straightforward Dockerfile. No need for intermediate hacks that we'd have to throw away in a month or two anyway.

1.) In most cases i currently build and cleanup inside the container. In that case its isolated without any further outside dependencies. The important part here is to install and cleanup in one RUN command to avoid hitting the cache layer to keep the container size small.

Yes, I see you are already doing that very diligently. :-)

2.) Another solution is to setup a proper CI pipeline (https://docs.travis-ci.com/user/languages/go/) and just copy the executable inside the container. I would only use such a solution if all steps are transparent to ensure that no malicious stuff finds a way inside the container and my network (with Travis which you use anyway this is possible).

Nah. I prefer what you currently have in Dockerfile, i.e. building from source, for maximum compatibility and security.

3.) We could also setup a Automatic Build Task on Docker Hub and define a branch which will trigger a new build with each code push. This also keep it transparent. You can find examples on my Docker Hub Account @ellerbrock, all my containers are build this way. I could help with the setup an and automation it if you like. Would be nice anyway to have an official Hugo Docker Container available.

That's a nice idea! What do you think, bep and @spf13?

[anthonyfok]

Hi @ellerbrock,

Thank you for drastically shrink the image from 277MB to 27MB! "Extreme" indeed!

You might like to consider applying this small change to further shrink the image to 20.5MB with the use of -ldflags '-s -w', see commit 9ed48c1

  -s	disable symbol table
  -w	disable DWARF generation

(I initially ran strip on it, but it turns out Go can omit those symbol table and DWARF all by itself.)

Cheers,
Anthony

[stale]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[ellerbrock]

hi @anthonyfok,

i would say we could close this issue, what do you think?

cheers maik

[stale]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.