How to install containerd with alpine-lima #489

afbjorklund · 2021-12-19T21:30:18Z

It's not fully clear how to add containerd to alpine, since the scripts require systemd:

ERRO[0018] [1 error occurred:
	* failed to satisfy the optional requirement 1 of 2 "systemd must be available": systemd is required to run containerd, but does not seem to be available.
Make sure that you use an image that supports systemd. If you do not want to run
containerd, please make sure that both 'container.system' and 'containerd.user'
are set to 'false' in the config file.

Installing containerd is quite simple, but packages for "nerdctl" and "buildkit" are missing...

provision:
  - mode: system
    script: |
      #!/bin/bash
      set -eux -o pipefail
      command -v containerd >/dev/null 2>&1 && exit 0
      apk add runc containerd cni-plugins
      sudo rc-update add containerd default
      sudo service containerd start

They can be added from the tarballs, but there doesn't seem to be any support available ?

That is, the current support is for nerdctl-full only (and not for nerdctl and buildkit archives)

Would it be better to make custom aports, or perhaps to add support for extra archives ?

containerd

nerdctl

nerdctl-0.15.0-linux-amd64.tar.gz
nerdctl-0.15.0-linux-arm64.tar.gz

buildkit

buildkit-v0.9.3.linux-amd64.tar.gz
buildkit-v0.9.3.linux-arm64.tar.gz
buildkit.confd
buildkit.initd

Both lima sudo nerdctl run and lima sudo nerdctl build seem to be doing just fine...

init-+-acpid
     |-7*[getty]
     |-sshd.pam---sshd.pam---sshd.pam-+-pstree
     |                                `-2*[sshfs---3*[{sshfs}]]
     |-supervise-daemo---lima-guestagent---7*[{lima-guestagent}]
     |-supervise-daemo---containerd---10*[{containerd}]
     |-supervise-daemo---buildkitd---8*[{buildkitd}]
     |-syslogd
     |-udevd
     `-udhcpc

The text was updated successfully, but these errors were encountered:

jandubois · 2021-12-19T21:51:48Z

There is some minimal support for adding nerdctl and buildkit from the nerdctl-full tarball in the image builder. It is used by the rd edition (Rancher Desktop), but does not include the openrc script to run buildkitd. It is trivial to define additional "editions" by creating a config file in the edition/ subdirectory.

jandubois · 2021-12-19T21:53:37Z

Forgot to add: the rd edition does not include containerd because we use the version bundled with k3s.

afbjorklund · 2021-12-20T06:23:50Z

Okay, so the same tarballs as above would work (or the full download) - just have to be done in the image builder.

I couldn't find any existing APKBUILD, but then again there doesn't seem to be any .deb or .rpm available either...

My main question was if it needed extending.

containerd:
  # Enable system-wide (aka rootful)  containerd and its dependencies (BuildKit, Stargz Snapshotter)
  # Default: false
  system: false
  # Enable user-scoped (aka rootless) containerd and its dependencies (currently requires systemd)
  # Default: true
  user: false
#  # Override containerd archive
#  # Default: hard-coded URL with hard-coded digest (see the output of `limactl info | jq .defaultTemplate.containerd.archives`)
#  archives:
#    - location: "~/Downloads/nerdctl-full-X.Y.Z-linux-amd64.tar.gz"
#      arch: "x86_64"
#      digest: "sha256:..."

extras:
    - name: "nerdctl"
      location: "https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-0.15.0-linux-amd64.tar.gz"
      arch: "x86_64"
      digest: "sha256:1371da3f6bd461f331946654f6dd3ef2ef4b9da0dd7bc5f78ed1166f32ad5adc"
      directory: "/usr/local/bin"
    - name: "nerdctl"
      location: "https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-0.15.0-linux-arm64.tar.gz"
      arch: "aarch64"
      digest: "sha256:7b79e2e8fd88b71ed4e0563c7e7dd27008b7ac7990ad2206efb012def850d150"
      directory: "/usr/local/bin"
    - name: "buildkit"
      location: "https://github.com/moby/buildkit/releases/download/v0.9.3/buildkit-v0.9.3.linux-amd64.tar.gz" 
      arch: "x86_64"
      digest: "sha256:f60461abdf2aee8444a4cb0607e4766da3bd503859320819ea8c43fe4a02576c"
      directory: "/usr/local"
    - name: "buildkit"
      location: "https://github.com/moby/buildkit/releases/download/v0.9.3/buildkit-v0.9.3.linux-arm64.tar.gz" 
      arch: "aarch64"
      digest: "sha256:3ee57ac33f8ff6ab1d187e25a217f8f2358826b14d707fd8fe0df6f536613aaf"
      directory: "/usr/local"

But that doesn't add the buildkit init script, either.

afbjorklund · 2022-01-03T12:29:55Z

I guess it needs some kind of "nerdctl" edition then, and then use that in the example instead of the "std" edition ?

https://github.com/lima-vm/alpine-lima#editions

afbjorklund · 2022-01-03T12:33:32Z

Added issue:

Create edition with nerdctl installed alpine-lima#34

If that is done, there is no need to make packages.

I couldn't find any existing APKBUILD, but then again there doesn't seem to be any .deb or .rpm available either...

It would still be "nice to have", but separate issue...

Make APKBUILD for buildkit alpine-lima#37

jandubois · 2022-01-03T18:00:04Z

I guess it needs some kind of "nerdctl" edition then, and then use that in the example instead of the "std" edition ?

It is not clear to me why this shouldn't just go into the std edition, which is the version that examples/lima.yaml is supposed to use. Is there a reason we need a separate edition for this?

afbjorklund · 2022-01-03T18:16:31Z

Is there a reason we need a separate edition for this?

I thought you wanted to avoid having containerd in std, so that it can be installed by lima (at runtime) or k3s ?

My original idea was just setting it to true, but it failed on systemd

containerd:
  system: true
  user: false

Systemd requirement for running rootless containers #310

jandubois · 2022-01-03T18:22:31Z

I thought you wanted to avoid having containerd in std, so that it can be installed by lima (at runtime) or k3s ?

I don't want it to run automatically, but I think having it pre-installed, together with the required initd scripts would be fine.

I'll reserve final judgement when I know how much it adds to the size of the ISO. 😄

A long time ago I was contemplating to extend the rootless setup script to be able to create openrc scripts in addition to systemd units. But I didn't really needed it, so it is kind of low on my todo pile...

afbjorklund · 2022-01-03T18:52:14Z

I'll reserve final judgement when I know how much it adds to the size of the ISO. smile

It seems the ISO is around 100 MB, so that makes it 1500 MB smaller than the "default"

555M	/home/anders/.lima/default/basedisk
851M	/home/anders/.lima/default/diffdisk

But from what I can tell, the previous build was 50 MB so maybe worth leaving out then ?

Also not having to include nerdctl-full does make the cidata.iso a lot smaller, if that counts.

jandubois · 2022-01-03T19:00:07Z

But from what I can tell, the previous build was 50 MB so maybe worth leaving out then ?

Maybe, but what is the use case for the std edition if it is no longer the standard version for Lima?

Also not having to include nerdctl-full does make the cidata.iso a lot smaller, if that counts.

Maybe; I'll have to think about this some more. IIRC @AkihiroSuda was somewhat opposed to bundling nerdctl with the ISO image, and prefers that it is always installed by lima. Which has the advantage of being able to update nerdctl without releasing a new iso. (The use case is different for Rancher Desktop, where you cannot upgrade lima, qemu, nerdctl etc. independent of RD anyways).

afbjorklund · 2022-01-03T19:03:47Z

Well, it does work - as long as you don't want to run containers on it ? Then it needs more software.

But it doesn't have to go in the ISO, it could be added from tgz or from apk - as originally sketched...

afbjorklund · 2022-01-03T19:16:58Z

A long time ago I was contemplating to extend the rootless setup script to be able to create openrc scripts in addition to systemd units. But I didn't really needed it, so it is kind of low on my todo pile...

It would be perfectly fine if rootless continued to fail, it was just for installing the daemons for use with sudo.

And installing from nerdctl/buildkit instead of nerdctl-full would be a nice feature, but it is not required either.

9,7M	nerdctl-0.15.0-linux-amd64.tar.gz
31M	containerd-1.5.8-linux-amd64.tar.gz
46M	buildkit-v0.9.3.linux-amd64.tar.gz

193M	nerdctl-full-0.15.0-linux-amd64.tar.gz

I would prefer if buildkit was also installed from apk, it seems to already be required* for containerd to use apk.

* Since alpine uses a different C library, only static binaries will work. The dynamic ones have glibc dependencies.

lima-vm/alpine-lima#34 (comment)

jandubois · 2022-01-10T19:24:38Z

Since alpine uses a different C library, only static binaries will work. The dynamic ones have glibc dependencies.

The buildkitd binary doesn't have glibc dependencies; we use the version from the nerdctl-full tarball (installed in the rd edition of the ISO) with Rancher Desktop, and it seems to work fine: rancher-sandbox/rancher-desktop#1182

Forgot to add: the rd edition does not include containerd because we use the version bundled with k3s.

This is actually not correct; the apk version of containerd is pulled in by docker-engine, but we don't actually use it (so far, it will be handy to run containerd/nerdctl without k3s in the future).

afbjorklund · 2022-01-10T19:27:32Z

There could be other reasons too, like the static tarball also bundles runc and qemu (already present in alpine)

bin/
bin/buildctl
bin/buildkit-qemu-aarch64
bin/buildkit-qemu-arm
bin/buildkit-qemu-i386
bin/buildkit-qemu-mips64
bin/buildkit-qemu-mips64el
bin/buildkit-qemu-ppc64le
bin/buildkit-qemu-riscv64
bin/buildkit-qemu-s390x
bin/buildkit-runc
bin/buildkitd

bin/buildkitd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=X2aFQsdVAvRxWnSawzAH/1Hw0V0nkfjHgYf9usyh3/fFM6G3pQuF8I1Z801d1I/7c0ogZwxeBYT3Q_A6vyA, not stripped

Or just building from source (for the sake of it).

jandubois · 2022-01-10T19:32:07Z

There could be other reasons too, like the static tarball also bundles runc and qemu (already present in alpine)

Not sure what you are trying to say; buildctl and buildkitd as well as a subset of the CNI plugins are all that we copy right now: https://github.com/lima-vm/alpine-lima/blob/v0.2.3/genapkovl-lima.sh#L214-L222

afbjorklund · 2022-01-10T19:32:40Z

I mean there could be other reasons from installing buildkit from an apk, sorry for being vague.

jandubois · 2022-01-10T19:41:22Z

This is kind of off-topic here, but the nerdctl-full distribution could be shrunk down a lot by stripping binaries and converting them to multi-call binaries, like busybox. Compare the k3s cni plugins:

lrwxrwxrwx 1 0 0    3 Aug 19 19:10 bridge -> cni
-rwxr-xr-x 1 0 0 3.1M Aug 19 19:09 cni
lrwxrwxrwx 1 0 0    3 Aug 19 19:10 flannel -> cni
lrwxrwxrwx 1 0 0    3 Aug 19 19:10 host-local -> cni
lrwxrwxrwx 1 0 0    3 Aug 19 19:10 loopback -> cni
lrwxrwxrwx 1 0 0    3 Aug 19 19:10 portmap -> cni

with the versions from (an older) nerdctl-full tarball (which was missing flannel):

-rwxr-xr-x  1 jan  staff   4.2M 11 Aug 08:46 bridge
-rwxr-xr-x  1 jan  staff   3.2M 11 Aug 08:46 host-local
-rwxr-xr-x  1 jan  staff   3.3M 11 Aug 08:46 loopback
-rwxr-xr-x  1 jan  staff   3.7M 11 Aug 08:46 portmap

There are a lot more cni plugins in the tarball, so the savings would add up, but I'm not sure if it is worth the effort.

afbjorklund · 2022-01-10T20:30:49Z

And the nerdctl-full download is fine (I wish that Podman had it, makes adoption much easier), just that it is also nice to be able to use apk add (or apt install) to install BuildKit instead of mucking about with architectures and versions and such.

afbjorklund · 2022-12-27T15:12:05Z

They are available as packages now, and a year has passed. The "alpine" image will still be bare bones.

AkihiroSuda added guest/alpine Guest: Alpine question Further information is requested labels Dec 20, 2021

afbjorklund mentioned this issue Jan 3, 2022

Create edition with nerdctl installed lima-vm/alpine-lima#34

Closed

afbjorklund mentioned this issue Jan 10, 2022

Allow installing system containerd for alpine #541

Closed

This comment has been minimized.

Sign in to view

afbjorklund closed this as not planned Won't fix, can't repro, duplicate, stale Dec 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to install containerd with alpine-lima #489

How to install containerd with alpine-lima #489

afbjorklund commented Dec 19, 2021 •

edited

Loading

jandubois commented Dec 19, 2021

jandubois commented Dec 19, 2021

afbjorklund commented Dec 20, 2021 •

edited

Loading

afbjorklund commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 •

edited

Loading

jandubois commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 •

edited

Loading

jandubois commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 •

edited

Loading

jandubois commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 •

edited

Loading

afbjorklund commented Jan 3, 2022 •

edited

Loading

jandubois commented Jan 10, 2022

afbjorklund commented Jan 10, 2022 •

edited

Loading

jandubois commented Jan 10, 2022

afbjorklund commented Jan 10, 2022

jandubois commented Jan 10, 2022

This comment has been minimized.

afbjorklund commented Jan 10, 2022

afbjorklund commented Dec 27, 2022

How to install containerd with alpine-lima #489

How to install containerd with alpine-lima #489

Comments

afbjorklund commented Dec 19, 2021 • edited Loading

jandubois commented Dec 19, 2021

jandubois commented Dec 19, 2021

afbjorklund commented Dec 20, 2021 • edited Loading

afbjorklund commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 • edited Loading

jandubois commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 • edited Loading

jandubois commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 • edited Loading

jandubois commented Jan 3, 2022

afbjorklund commented Jan 3, 2022 • edited Loading

afbjorklund commented Jan 3, 2022 • edited Loading

jandubois commented Jan 10, 2022

afbjorklund commented Jan 10, 2022 • edited Loading

jandubois commented Jan 10, 2022

afbjorklund commented Jan 10, 2022

jandubois commented Jan 10, 2022

This comment has been minimized.

afbjorklund commented Jan 10, 2022

afbjorklund commented Dec 27, 2022

afbjorklund commented Dec 19, 2021 •

edited

Loading

afbjorklund commented Dec 20, 2021 •

edited

Loading

afbjorklund commented Jan 3, 2022 •

edited

Loading

afbjorklund commented Jan 3, 2022 •

edited

Loading

afbjorklund commented Jan 3, 2022 •

edited

Loading

afbjorklund commented Jan 3, 2022 •

edited

Loading

afbjorklund commented Jan 3, 2022 •

edited

Loading

afbjorklund commented Jan 10, 2022 •

edited

Loading