docker: reduce container image size

[stapelberg]

Current size is 1.07 GB uncompressed, 324 MB compressed. docker or the docker hub seems to transparently compress, so the progress output shows me that I’m uploading 1.07 GB, but when downloading the container image from the docker hub, I only need to download 324 MB.

I’m filing this so that we can track ideas, but I don’t plan to work on this actively right now. It’s unfortunate that the container is so large.

[jojo243]

Maybe if you provide the Dockerfile you are using, I can look into this (I have some experience converting Applications into docker images of minimal size)

[stapelberg]

I’m not using a Dockerfile. I’m creating a root file system via

distri/cmd/distri/pack.go

Line 174 in 1d68a13

if p.docker {

, which is directly packed into a single-layer container.

The usual techniques of optimizing Dockerfiles won’t help here.

[pwaller]

You don't mention in your issue how big it is now, which might be an interesting thing to have track of :)

(Nice effort btw!)

[stapelberg]

Update the first comment on this issue to include the size. I suppose transferring 324 MB isn’t so bad actually.

[pwaller]

Yup, put that way it doesn't sound bad :)

[pgaskin]

As of supersilverhaze, the majority of the disk usage comes from locale-archive in glibc and the kernel modules in linux. Both of these are unnecessary in Docker images. locale-archive can be shrunk using localedef, and the Linux kernel isn't needed at all.

In addition grub2-efi and containerd aren't necessary either.

I'm still relatively new to distri (I noticed it a while ago, but I'm only looking at in depth now), so I'm not completely familiar with the build system yet.

[stapelberg]

Thanks for taking a look! For glibc locales, I had recently stumbled upon https://fedoraproject.org/wiki/QA:Glibc_locale_subpackaging, but haven’t investigated more details or how to adopt that strategy in distri. If you wanted to look into it, that’d be cool.

[pgaskin]

I had a look at that, and the overall concept seems pretty simple. Basically, you just patch glibc's Makefile to add --no-archive to the localedef command line in define build-one-locale, then you can take the individual locales as folders in {prefix}/lib/locale. To generate custom ones, it's just localedef --no-archive -i {locale} -f {encoding}, and you can find it in {prefix}/lib/locale/{locale}.{encoding}.

To do this, you'd need package generation, or it's not really feasible (unless you choose to ship all locales by default, but have a few separate packages for common ones like English)`. I'll have a look at distri later today.

[pgaskin]

I noticed that in here:

distri/pkgs/base/build.textproto

Lines 17 to 18 in 434b729

	# TODO: split out into base-boot once pack supports images that don’t need
	# booting (e.g. docker)

, you were planning to split the packages into base-boot. Should this be done now?

[stapelberg]

To do this, you'd need package generation, or it's not really feasible (unless you choose to ship all locales by default, but have a few separate packages for common ones like English)`. I'll have a look at distri later today.

Having a common english locale by default and the others in an extra package sounds good to me.

, you were planning to split the packages into base-boot. Should this be done now?

Sure, why not :)

[pgaskin]

Sure, why not :)

What should be done with the linux and containerd packages?

P.S. I won't be able to get around to actually implementing and testing the locale stuff until next week.

[stapelberg]

What should be done with the linux and containerd packages?

How do you mean? They should only be depended on from the base packages that don’t go into the docker container. Is that what you mean, or did you have a different question?

[pgaskin]

How do you mean?

For containerd, I'm not completely sure what the TODO is referring to.:

distri/pkgs/base/build.textproto

Lines 30 to 31 in 52cce77

	# TODO: remove once pack no longer unconditionally enables units:
	runtime_dep: "containerd"

For linux, I just wanted to confirm that it isn't needed by anything else implicitly (the comment about base-boot is on the other block):

distri/pkgs/base/build.textproto

Lines 17 to 28 in 52cce77

	# TODO: split out into base-boot once pack supports images that don’t need
	# booting (e.g. docker)
	runtime_dep: "systemd" # for booting
	runtime_dep: "dbus" # for e.g. systemd’s timedatectl
	runtime_dep: "grub2" # for BIOS boot
	runtime_dep: "grub2-efi" # for UEFI boot
	# For creating an initramfs:
	runtime_dep: "dracut"
	runtime_dep: "linux"
	runtime_dep: "linux-firmware"
	runtime_dep: "cryptsetup" # for pack -encrypt

[stapelberg]

For containerd, I'm not completely sure what the TODO is referring to.:

This is referring to:

distri/cmd/distri/pack.go

Lines 503 to 522 in 52cce77

	// TODO: dynamically find which units to enable (test: xdm)
	units := []string{
	"systemd-networkd",
	"containerd",
	"docker",
	"ssh",
	"haveged",
	}
	if p.extraBase == "base-x11" {
	units = append(units, "debugfs", "srcfs")
	}
	cmd = exec.Command("unshare",
	append([]string{
	"--user",
	"--map-root-user", // for mount permissions in the namespace
	"--mount",
	"--",
	"chroot", root, "/ro/bin/systemctl",
	"enable",
	}, units...)...)

For linux, I just wanted to confirm that it isn't needed by anything else implicitly (the comment about base-boot is on the other block):

If something else needs the linux package, it should have a dependency declared on it, no? :)

Don’t worry about breaking things too much. We can always revert or fix later.