Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker: reduce container image size #28

Open
stapelberg opened this issue Aug 7, 2019 · 14 comments
Open

docker: reduce container image size #28

stapelberg opened this issue Aug 7, 2019 · 14 comments

Comments

@stapelberg
Copy link
Member

@stapelberg stapelberg commented Aug 7, 2019

Current size is 1.07 GB uncompressed, 324 MB compressed. docker or the docker hub seems to transparently compress, so the progress output shows me that I’m uploading 1.07 GB, but when downloading the container image from the docker hub, I only need to download 324 MB.

I’m filing this so that we can track ideas, but I don’t plan to work on this actively right now. It’s unfortunate that the container is so large.

@jojo243
Copy link

@jojo243 jojo243 commented Jan 20, 2020

Maybe if you provide the Dockerfile you are using, I can look into this (I have some experience converting Applications into docker images of minimal size)

@stapelberg
Copy link
Member Author

@stapelberg stapelberg commented Jan 20, 2020

I’m not using a Dockerfile. I’m creating a root file system via

if p.docker {
, which is directly packed into a single-layer container.

The usual techniques of optimizing Dockerfiles won’t help here.

@pwaller
Copy link

@pwaller pwaller commented Jan 22, 2020

You don't mention in your issue how big it is now, which might be an interesting thing to have track of :)

(Nice effort btw!)

@stapelberg
Copy link
Member Author

@stapelberg stapelberg commented Jan 22, 2020

Update the first comment on this issue to include the size. I suppose transferring 324 MB isn’t so bad actually.

@pwaller
Copy link

@pwaller pwaller commented Jan 23, 2020

Yup, put that way it doesn't sound bad :)

@pgaskin
Copy link

@pgaskin pgaskin commented May 28, 2020

As of supersilverhaze, the majority of the disk usage comes from locale-archive in glibc and the kernel modules in linux. Both of these are unnecessary in Docker images. locale-archive can be shrunk using localedef, and the Linux kernel isn't needed at all.

In addition grub2-efi and containerd aren't necessary either.

I'm still relatively new to distri (I noticed it a while ago, but I'm only looking at in depth now), so I'm not completely familiar with the build system yet.

@stapelberg
Copy link
Member Author

@stapelberg stapelberg commented May 28, 2020

Thanks for taking a look! For glibc locales, I had recently stumbled upon https://fedoraproject.org/wiki/QA:Glibc_locale_subpackaging, but haven’t investigated more details or how to adopt that strategy in distri. If you wanted to look into it, that’d be cool.

@pgaskin
Copy link

@pgaskin pgaskin commented May 28, 2020

I had a look at that, and the overall concept seems pretty simple. Basically, you just patch glibc's Makefile to add --no-archive to the localedef command line in define build-one-locale, then you can take the individual locales as folders in {prefix}/lib/locale. To generate custom ones, it's just localedef --no-archive -i {locale} -f {encoding}, and you can find it in {prefix}/lib/locale/{locale}.{encoding}.

To do this, you'd need package generation, or it's not really feasible (unless you choose to ship all locales by default, but have a few separate packages for common ones like English)`. I'll have a look at distri later today.

@pgaskin
Copy link

@pgaskin pgaskin commented May 28, 2020

I noticed that in here:

# TODO: split out into base-boot once pack supports images that don’t need
# booting (e.g. docker)

, you were planning to split the packages into base-boot. Should this be done now?

@stapelberg
Copy link
Member Author

@stapelberg stapelberg commented May 28, 2020

To do this, you'd need package generation, or it's not really feasible (unless you choose to ship all locales by default, but have a few separate packages for common ones like English)`. I'll have a look at distri later today.

Having a common english locale by default and the others in an extra package sounds good to me.

, you were planning to split the packages into base-boot. Should this be done now?

Sure, why not :)

@pgaskin
Copy link

@pgaskin pgaskin commented May 28, 2020

Sure, why not :)

What should be done with the linux and containerd packages?

P.S. I won't be able to get around to actually implementing and testing the locale stuff until next week.

@stapelberg
Copy link
Member Author

@stapelberg stapelberg commented May 29, 2020

What should be done with the linux and containerd packages?

How do you mean? They should only be depended on from the base packages that don’t go into the docker container. Is that what you mean, or did you have a different question?

@pgaskin
Copy link

@pgaskin pgaskin commented Jun 8, 2020

How do you mean?

For containerd, I'm not completely sure what the TODO is referring to.:

# TODO: remove once pack no longer unconditionally enables units:
runtime_dep: "containerd"

For linux, I just wanted to confirm that it isn't needed by anything else implicitly (the comment about base-boot is on the other block):

# TODO: split out into base-boot once pack supports images that don’t need
# booting (e.g. docker)
runtime_dep: "systemd" # for booting
runtime_dep: "dbus" # for e.g. systemd’s timedatectl
runtime_dep: "grub2" # for BIOS boot
runtime_dep: "grub2-efi" # for UEFI boot
# For creating an initramfs:
runtime_dep: "dracut"
runtime_dep: "linux"
runtime_dep: "linux-firmware"
runtime_dep: "cryptsetup" # for pack -encrypt

@stapelberg
Copy link
Member Author

@stapelberg stapelberg commented Jun 10, 2020

For containerd, I'm not completely sure what the TODO is referring to.:

This is referring to:

distri/cmd/distri/pack.go

Lines 503 to 522 in 52cce77

// TODO: dynamically find which units to enable (test: xdm)
units := []string{
"systemd-networkd",
"containerd",
"docker",
"ssh",
"haveged",
}
if p.extraBase == "base-x11" {
units = append(units, "debugfs", "srcfs")
}
cmd = exec.Command("unshare",
append([]string{
"--user",
"--map-root-user", // for mount permissions in the namespace
"--mount",
"--",
"chroot", root, "/ro/bin/systemctl",
"enable",
}, units...)...)

For linux, I just wanted to confirm that it isn't needed by anything else implicitly (the comment about base-boot is on the other block):

If something else needs the linux package, it should have a dependency declared on it, no? :)

Don’t worry about breaking things too much. We can always revert or fix later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants