Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker: reduce container image size #28

Open
stapelberg opened this issue Aug 7, 2019 · 14 comments
Open

docker: reduce container image size #28

stapelberg opened this issue Aug 7, 2019 · 14 comments
Labels
enhancement New feature or request

Comments

@stapelberg
Copy link
Member

stapelberg commented Aug 7, 2019

Current size is 1.07 GB uncompressed, 324 MB compressed. docker or the docker hub seems to transparently compress, so the progress output shows me that I’m uploading 1.07 GB, but when downloading the container image from the docker hub, I only need to download 324 MB.

I’m filing this so that we can track ideas, but I don’t plan to work on this actively right now. It’s unfortunate that the container is so large.

@stapelberg stapelberg added the enhancement New feature or request label Aug 7, 2019
@jojo243
Copy link

jojo243 commented Jan 20, 2020

Maybe if you provide the Dockerfile you are using, I can look into this (I have some experience converting Applications into docker images of minimal size)

@stapelberg
Copy link
Member Author

I’m not using a Dockerfile. I’m creating a root file system via

if p.docker {
, which is directly packed into a single-layer container.

The usual techniques of optimizing Dockerfiles won’t help here.

@pwaller
Copy link

pwaller commented Jan 22, 2020

You don't mention in your issue how big it is now, which might be an interesting thing to have track of :)

(Nice effort btw!)

@stapelberg
Copy link
Member Author

Update the first comment on this issue to include the size. I suppose transferring 324 MB isn’t so bad actually.

@pwaller
Copy link

pwaller commented Jan 23, 2020

Yup, put that way it doesn't sound bad :)

@pgaskin
Copy link

pgaskin commented May 28, 2020

As of supersilverhaze, the majority of the disk usage comes from locale-archive in glibc and the kernel modules in linux. Both of these are unnecessary in Docker images. locale-archive can be shrunk using localedef, and the Linux kernel isn't needed at all.

In addition grub2-efi and containerd aren't necessary either.

I'm still relatively new to distri (I noticed it a while ago, but I'm only looking at in depth now), so I'm not completely familiar with the build system yet.

@stapelberg
Copy link
Member Author

Thanks for taking a look! For glibc locales, I had recently stumbled upon https://fedoraproject.org/wiki/QA:Glibc_locale_subpackaging, but haven’t investigated more details or how to adopt that strategy in distri. If you wanted to look into it, that’d be cool.

@pgaskin
Copy link

pgaskin commented May 28, 2020

I had a look at that, and the overall concept seems pretty simple. Basically, you just patch glibc's Makefile to add --no-archive to the localedef command line in define build-one-locale, then you can take the individual locales as folders in {prefix}/lib/locale. To generate custom ones, it's just localedef --no-archive -i {locale} -f {encoding}, and you can find it in {prefix}/lib/locale/{locale}.{encoding}.

To do this, you'd need package generation, or it's not really feasible (unless you choose to ship all locales by default, but have a few separate packages for common ones like English)`. I'll have a look at distri later today.

@pgaskin
Copy link

pgaskin commented May 28, 2020

I noticed that in here:

# TODO: split out into base-boot once pack supports images that don’t need
# booting (e.g. docker)

, you were planning to split the packages into base-boot. Should this be done now?

@stapelberg
Copy link
Member Author

To do this, you'd need package generation, or it's not really feasible (unless you choose to ship all locales by default, but have a few separate packages for common ones like English)`. I'll have a look at distri later today.

Having a common english locale by default and the others in an extra package sounds good to me.

, you were planning to split the packages into base-boot. Should this be done now?

Sure, why not :)

@pgaskin
Copy link

pgaskin commented May 28, 2020

Sure, why not :)

What should be done with the linux and containerd packages?

P.S. I won't be able to get around to actually implementing and testing the locale stuff until next week.

@stapelberg
Copy link
Member Author

What should be done with the linux and containerd packages?

How do you mean? They should only be depended on from the base packages that don’t go into the docker container. Is that what you mean, or did you have a different question?

@pgaskin
Copy link

pgaskin commented Jun 8, 2020

How do you mean?

For containerd, I'm not completely sure what the TODO is referring to.:

# TODO: remove once pack no longer unconditionally enables units:
runtime_dep: "containerd"

For linux, I just wanted to confirm that it isn't needed by anything else implicitly (the comment about base-boot is on the other block):

# TODO: split out into base-boot once pack supports images that don’t need
# booting (e.g. docker)
runtime_dep: "systemd" # for booting
runtime_dep: "dbus" # for e.g. systemd’s timedatectl
runtime_dep: "grub2" # for BIOS boot
runtime_dep: "grub2-efi" # for UEFI boot
# For creating an initramfs:
runtime_dep: "dracut"
runtime_dep: "linux"
runtime_dep: "linux-firmware"
runtime_dep: "cryptsetup" # for pack -encrypt

@stapelberg
Copy link
Member Author

For containerd, I'm not completely sure what the TODO is referring to.:

This is referring to:

distri/cmd/distri/pack.go

Lines 503 to 522 in 52cce77

// TODO: dynamically find which units to enable (test: xdm)
units := []string{
"systemd-networkd",
"containerd",
"docker",
"ssh",
"haveged",
}
if p.extraBase == "base-x11" {
units = append(units, "debugfs", "srcfs")
}
cmd = exec.Command("unshare",
append([]string{
"--user",
"--map-root-user", // for mount permissions in the namespace
"--mount",
"--",
"chroot", root, "/ro/bin/systemctl",
"enable",
}, units...)...)

For linux, I just wanted to confirm that it isn't needed by anything else implicitly (the comment about base-boot is on the other block):

If something else needs the linux package, it should have a dependency declared on it, no? :)

Don’t worry about breaking things too much. We can always revert or fix later.

@stapelberg stapelberg mentioned this issue Nov 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants