Look for ways to Reduce Image Size

[LongLiveCHIEF]

Current size of main image is clocking in at 1GB. The base image we are building from starts at around 350Mb. Let's see if we can find ways to trim the fat.

[LongLiveCHIEF]

What's interesting, is I just built the minimal image (#156) and even that image is clocking in at just under < 800Mb. I'm afraid of cutting too much, since octoprint will need system libs for installing and using plugins, and because users of this image typically use it more like a VM than a docker image, and removing common utilities and packages will just wind up fracturing the docker community.

[torresmvl]

Hi, i've started to develop an octoprint image from alpine and i've managed to reduce its image size to 133MB to the bare minimal packages needed for octoprint to run. Plugins are installed and managed with pip without issues. There is a reason to the stack (octoprint, mjpg-streamer, haproxy) are packaged inside the image? Couldn't be deployed on docker-compose.yml

version: '3'

services:
  haproxy:
  octoprint-alpine:
  mjpg-streamer:

ps: hadn't plugged and tested any printer or webcam at this point

[LongLiveCHIEF]

Have you seen our minimal image? This issue is specific to the main image, although there are steps we can take to further optimize the minimal image, and we'd accept any PR's you have.

https://github.com/OctoPrint/octoprint-docker/tree/master/minimal
https://github.com/OctoPrint/octoprint-docker/blob/master/docs/using_the_minimal_image.md

[torresmvl]

Ok, i see. There's some interest for an alpine branch tag? I'm keeping up @nunofgs works; will test over the weekend.
Will also dive inside main and minimal images.

[LongLiveCHIEF]

@torresmvl yes. And @nunofgs image was deprecated when v1 of this image was dropped. He worked with us to ensure compatibility, but he's no longer maintaining that image.

[torresmvl]

Inspecting the image inside my local portainer deployment, it seems that this step is weightening 590MB uncompressed (although it weighs less than 300MB on Docker Hub image registries)

RUN apt-get update && apt-get install -y \
  avrdude \
  build-essential \
  cmake \
  curl \
  imagemagick \
  ffmpeg \
  fontconfig \
  g++ \
  git \
  haproxy \
  libjpeg-dev \
  libjpeg62-turbo \
  libprotobuf-dev \
  libv4l-dev \
  openssh-client \
  v4l-utils \
  xz-utils \
  zlib1g-dev

I'm not sure why some packages are being installed (such as avrdude). Some packages are only needed for the building stage, and not for the runtime and it seems that every recommended package are being installed too. Will get my hands dirty over the week.

[torresmvl]

I've tested my alpine build past weekend and it was sucessful. Tested with RPi v1.2 and RPi 4 and within a x86 ubuntu VM. i'll begin to make it compatible with this repo folder structure and will be commiting at growlab-digital/octoprint-docker untill compatibility be satisfied. Images can be pulled from Docker Hub: growlab/octoprint-alpine.
Alert: hadn't tested webcam since i don't have one, but it should be working fine.

[LongLiveCHIEF]

Inspecting the image inside my local portainer deployment, it seems that this step is weightening 590MB uncompressed (although it weighs less than 300MB on Docker Hub image registries)

RUN apt-get update && apt-get install -y \
  avrdude \
  build-essential \
  cmake \
  curl \
  imagemagick \
  ffmpeg \
  fontconfig \
  g++ \
  git \
  haproxy \
  libjpeg-dev \
  libjpeg62-turbo \
  libprotobuf-dev \
  libv4l-dev \
  openssh-client \
  v4l-utils \
  xz-utils \
  zlib1g-dev

I'm not sure why some packages are being installed (such as avrdude). Some packages are only needed for the building stage, and not for the runtime and it seems that every recommended package are being installed too. Will get my hands dirty over the week.

yeah, taking a close look at each and every package installed in that command is definitely something we need to do with this cleanup.

[torresmvl]

yeah, taking a close look at each and every package installed in that command is definitely something we need to do with this cleanup.

This can guide us

root@octoprint:/# du -hd 1 /
0       /dev
11M     /bin
8.0K    /run
20M     /root
4.0K    /boot
0       /sys
0       /proc
2.7M    /etc
4.0K    /mnt
4.0K    /srv
22M     /opt
35M     /var
15M     /lib
4.0K    /media
821M    /usr
4.0K    /home
4.2M    /sbin
4.0K    /lib64
1.8M    /tmp
16K     /octoprint
5.0M    /mjpg
92K     /libexec
942M    /

[LongLiveCHIEF]

That means nothing. It's a union file system, thus that command shows things from all layers (even the ones we inherit from).

[ondrovic]

A lot of the bloat of this image is due to the build tools that are installed using the following two commands:

663 mb - RUN|1 octoprint_ref=1.5.3 /bin/sh -c apt-get update && apt-get install -y   avrdude   build-essential cmake   curl   imagemagick   ffmpeg   fontconfig   g++   git   haproxy   libjpeg-dev   libjpeg62-turbo libprotobuf-dev   libv4l-dev   openssh-client   v4l-utils   xz-utils   zlib1g-dev # buildkit

105 mb - RUN |1 octoprint_ref=1.5.3 /bin/sh -c pip install . # buildkit

If you could have a multi stage image, that builds the instance, then just copy out the compiled files of what you need it would greatly reduce the image size. We do something similar with our production builds of our react apps. I would assume you could do the same thing with this.

In short it's spec like this

• Base image
• Builder Image - has all the build tools and libraries installed, builds
• Final Build - copies the complied source from the builder and is responsible for running the final image

multistage-builds

[LongLiveCHIEF]

I'm familiar with multi-stage builds (we're using them). The problem is that many of the things you're considering as build tools are used by several common and large components of the octoprint ecosystem at runtime, not just build time.

We had removed many of these in a patch just 2 weeks ago, and had to revert almost right away due to issues.

At the moment, our biggest holdup in reducing image size is a a good e2e testing automation pipeline, which will allow us to remove packages with confidence.

[AdrianoKF]

I haven't tried it out yet, but it seems like cleaning apt's cache and list files after installing packages would be a quick win to reduce image size by a few MB (as suggested also by the official Dockerfile best practices):

RUN apt-get update && apt-get install -y \
    package-bar \
    package-baz \
    package-foo  \
    && rm -rf /var/lib/apt/lists/*

Additional savings could be had by running apt-get clean as well in the same RUN statement to remove unused .deb files from the apt cache.

If it would be interesting to you, I can follow up and prepare a PR, measuring the effect of these changes.

[LongLiveCHIEF]

@AdrianoKF sure, that would be great. I haven't gotten around to it yet myself so that would def help. It won't make much of a difference because this image is particularly huge, but every little bit helps!

[mley]

@torresmvl Is there any chance we get your Dockerfile of the Alpine build? The linked repository is gone unfortunately.

[bradtho]

I know this issue has been open for a few years now but I've been actively investigating and testing the creation of an Alpine image.

Ideally with a view to it being run off of Kubernetes so I would be baking in Octoprint-Prometheus-Exporter and hopefully an Open-Telemetry plugin too.

If anyone could point me towards the directories/binaries that are required at runtime as opposed to build that'd be amazing! 🙏🏻 and if there's enough interest then I'm happy to share my work.