Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite Dockerfile to use Alpine Linux, and produce a minimal image #2004

Closed
wants to merge 2 commits into from

Conversation

thaJeztah
Copy link

This pull request contains two changes (described below). With this change in place, it should be possible to enable auto-builds on Docker Hub, as it looks like the current image has been last updated four years ago, is likely outdated, and probably contains vulnerabilities;

Screenshot 2019-11-01 at 11 12 26

Change Dockerfile to be a multistage build

The existing Dockerfile produced a broken binary (static
compile caused the binary to segfault).

This patch changes the Dockerfile to

  • Build a non-static binary

  • Use a multi-stage build: separate stages are used for build and
    deploy; the build stage contains the tools needed for building
    the binary, after which the build artefacts are copied to the
    final stage. which produces the actual image.
    Using this approach, the build-tools can be kept in the Docker
    build-cache, which improves performance of repeated builds and
    does not require cleanup steps in the Dockerfile itself, which
    can simplify the steps.

  • The official Python image is used for the build stage, so that
    there is no need to install python manually. The version of python
    is set to the same version as the version used in CI, but can be
    overridden using at build-time using a build-arg:

    docker build --build-arg PYTHON_VERSION=x ..
  • Both build- and deploy stages use the debian "stretch" variants,
    which matches what's used in the current version of the Dockerfile,
    but could be updated to the current stable ("buster") variant if
    there is a need.

  • The final stage uses the "slim" variant to reduce the size. For
    comparison, below are sizes of the final image when using the
    regular, or "slim" version of the image:

    REPOSITORY  TAG                 IMAGE ID            CREATED             SIZE
    jq          stretch-slim        4b7f380970f0        About a minute ago  70.3MB
    jq          stretch             c5fa8b766cd4        7 minutes ago       119MB

A .dockerignore file is also added to prevent "busting" the Docker
build-cache if not needed, and steps in the Dockerfile are optimized
for BuildKit, but remain compatible with the legacy builder. To build,
and verify the image:

DOCKER_BUILDKIT=1 docker build -t jq .

And, test the image, use it (for example) to pretty-print its own
"docker image inspect" output:

docker image inspect jq | docker run -i --rm jq .
[
  {
    "Id": "sha256:4b7f380970f0d06d4ca9137fbbb3fa3f6554da996147d1d956cda6f44be25650",
    "RepoTags": [
      "jq:latest",
      "jq:stretch-slim"
    ],

Dockerfile: use Alpine image to build static binary

This patch switches the Dockerfile to use an Alpine Linux
base image to build the static binary. Given that the binary
is now fully static, a minimal image can be produced that
only contains the essential artefacts.

The static version of the Docker image does not contain the man-
pages, so the existing (debian based) Dockerfile is kept, but
renamed to Dockerfile.debian, and can be built by specifying
the alternative Dockerfile using the -f flag:

DOCKER_BUILDKIT=1 docker build -t jq:stretch -f Dockerfile.debian .

With this patch applied, the image size is reduced drastically:

DOCKER_BUILDKIT=1 docker build -t jq .

docker image ls

REPOSITORY  TAG                 IMAGE ID            CREATED             SIZE
jq          latest              7da2ea1c016d        5 seconds ago       800kB
jq          stretch-slim        4b7f380970f0        About a minute ago  70.3MB
jq          stretch             c5fa8b766cd4        7 minutes ago       119MB

The image can be tested, for example, by inspecting itself and pretty-
printing the json:

docker image inspect jq | docker run -i --rm jq .
[
  {
    "Id": "sha256:7da2ea1c016d97e7b52b09d5a323a1e7c0e4dbdbc77ce2715aedd3188721e4da",
    "RepoTags": [
      "jq:latest",
      "jq:static"
    ],
...

Or a simple test;

echo '{"a":1, "b":2}' | docker run -i --rm jq '.a+.b'

@coveralls
Copy link

coveralls commented Nov 1, 2019

Coverage Status

Coverage remained the same at 84.134% when pulling 0e98fdf on thaJeztah:multistage_dockerfile into a17dd32 on stedolan:master.

@thaJeztah
Copy link
Author

ping @stedolan @wtlangford ptal

@Nicceboy
Copy link

Any updates about this? The image size change alone would make this really useful.

@kolyshkin
Copy link

@wtlangford @nicowilliams PTAL

Dockerfile Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
The existing Dockerfile produced a broken binary (static
compile caused the binary to segfault).

This patch changes the Dockerfile to

- Build a non-static binary
- Use a multi-stage build: separate stages are used for build and
  deploy; the build stage contains the tools needed for building
  the binary, after which the build artefacts are copied to the
  final stage. which produces the actual image.
  Using this approach, the build-tools can be kept in the Docker
  build-cache, which improves performance of repeated builds and
  does not require cleanup steps in the Dockerfile itself, which
  can simplify the steps.
- The official Python image is used for the build stage, so that
  there is no need to install python manually. The version of python
  is set to the same version as the version used in CI, but can be
  overridden using at build-time using a build-arg:

    docker build --build-arg PYTHON_VERSION=x ..

- Both build- and deploy stages use the debian "stretch" variants,
  which matches what's used in the current version of the Dockerfile,
  but could be updated to the current stable ("buster") variant if
  there is a need.
- The final stage uses the "slim" variant to reduce the size. For
  comparison, below are sizes of the final image when using the
  regular, or "slim" version of the image:

    REPOSITORY  TAG                 IMAGE ID            CREATED             SIZE
    jq          stretch-slim        4b7f380970f0        About a minute ago  70.3MB
    jq          stretch             c5fa8b766cd4        7 minutes ago       119MB

A .dockerignore file is also added to prevent "busting" the Docker
build-cache if not needed, and steps in the Dockerfile are optimized
for BuildKit, but remain compatible with the legacy builder. To build,
and verify the image:

    DOCKER_BUILDKIT=1 docker build -t jq .

And, test the image, us it (for example) to pretty-print its own
"docker image inspect" output:

    docker image inspect jq | docker run -i --rm jq .
    [
      {
        "Id": "sha256:4b7f380970f0d06d4ca9137fbbb3fa3f6554da996147d1d956cda6f44be25650",
        "RepoTags": [
          "jq:latest",
          "jq:stretch-slim"
        ],

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch switches the Dockerfile to use an Alpine Linux
base image to build the static binary. Given that the binary
is now fully static, a minimal image can be produced that
only contains the essential artefacts.

The static version of the Docker image does not contain the man-
pages, so the existing (debian based) Dockerfile is kept, but
renamed to Dockerfile.debian, and can be built by specifying
the alternative Dockerfile using the `-f` flag:

    DOCKER_BUILDKIT=1 docker build -t jq:stretch -f Dockerfile.debian .

With this patch applied, the image size is reduced drastically:

    DOCKER_BUILDKIT=1 docker build -t jq .

    docker image ls

    REPOSITORY  TAG                 IMAGE ID            CREATED             SIZE
    jq          latest              7da2ea1c016d        5 seconds ago       800kB
    jq          stretch-slim        4b7f380970f0        About a minute ago  70.3MB
    jq          stretch             c5fa8b766cd4        7 minutes ago       119MB

The image can be tested, for example, by inspecting itself and pretty-
printing the json:

    docker image inspect jq | docker run -i --rm jq .
    [
      {
        "Id": "sha256:7da2ea1c016d97e7b52b09d5a323a1e7c0e4dbdbc77ce2715aedd3188721e4da",
        "RepoTags": [
          "jq:latest",
          "jq:static"
        ],
    ...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Author

@stedolan @kolyshkin @tao12345666333 forgot about this one; updated with your recommendations; PTAL

Copy link

@tao12345666333 tao12345666333 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants