Docker for Desktop

[errordeveloper]

Proposal / RFE

Is your feature request related to a problem?

It would be nice if one could use Docker for Mac and Windows to try out Cilium.

Describe the solution you'd like

A user guide would be nice.

[errordeveloper]

From current stable version of Docker for Mac (2.2.03 - Docker Engine v19.03.5, Kubernetes v1.15.5):

$ docker run --privileged --tty --interactive --pid=host alpine nsenter -t 1 -m
610840c9932f:/# uname -a
Linux 610840c9932f 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 Linux
610840c9932f:/# zcat /proc/config.gz  | grep -i bpf
CONFIG_CGROUP_BPF=y
CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_NETFILTER_XT_MATCH_BPF=y
CONFIG_BPFILTER=y
CONFIG_BPFILTER_UMH=m
CONFIG_NET_CLS_BPF=y
CONFIG_NET_ACT_BPF=y
CONFIG_BPF_JIT=y
# CONFIG_BPF_STREAM_PARSER is not set
CONFIG_LWTUNNEL_BPF=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_BPF_EVENTS=y
# CONFIG_BPF_KPROBE_OVERRIDE is not set
# CONFIG_TEST_BPF is not set
610840c9932f:/# 

[lrouquette]

Awesome idea! In the same spirit, I'd love to be able to use this with kind.

[errordeveloper]

Just assigned to myself, as I am already looking into this.

/xref docker/for-mac#4454 linuxkit/linuxkit#3516

[skurfuerst]

Hey @errordeveloper @lrouquette ,

I figured out how you can run Cilium with k3s / k3d on docker for mac. See https://sandstorm.de/de/blog/post/running-cilium-in-k3s-and-k3d-lightweight-kubernetes-on-mac-os-for-development.html for the full explanation :-) Feel free to re-use the content for your documentation or so :-)

All the best,
and keep up the great work,
Sebastian

[lrouquette]

Hey @errordeveloper @lrouquette ,

I figured out how you can run Cilium with k3s / k3d on docker for mac. See https://sandstorm.de/de/blog/post/running-cilium-in-k3s-and-k3d-lightweight-kubernetes-on-mac-os-for-development.html for the full explanation :-) Feel free to re-use the content for your documentation or so :-)

All the best,
and keep up the great work,
Sebastian

Nice work @skurfuerst ! I'll give it a try!

[errordeveloper]

@skurfuerst it's good to hear you have this working with k3d, I also heard folks made it work with kind. My original intention was to make Cilium work on Docker for Desktop with built-in version of Kubernetes and without node-init hacks. I can see you have mentioned in your blog post that node-init didn't work as expected due to to dependency on /bin/bash, that to me sounds like a trivial bug in node-init that can be fixed, please be sure to open another issue for that.

[skurfuerst]

@errordeveloper awesome :) #14489 here it is!

[Forbzy]

@errordeveloper This is exactly what I was looking for. I'm using Docker Desktop for Windows 3.2.2
I hope to see this new feature soon.

[github-actions]

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

[joestringer]

I think that some contributors have been using Docker for Desktop or even LIMA recently. I'll leave the issue open though as it would still be neat to include a guide in the repository / docs somewhere to help new contributors.

[angofanguishire]

@skurfuerst I followed your tutorial step by step to test out Cilium on Mac.

I am using MacBook Pro
System version: 12.2.1
Kernel version: Darwin 21.3.0
Chip: Apple M1 pro
CPU: 10
Memory: 16GB

Docker version:

K3D version: v5.4.6
K3S version: v1.24.4-k3s1 (default)

Helm version: version.BuildInfo{Version:"v3.9.4", GitCommit:"dbc6d8e20fe1d58d50e6ed30f09a04a77e4c68db", GitTreeState:"clean", GoVersion:"go1.19"}

The steps I took are as follows:

1. Created a k3d cluster with 1 server and 2 agents using the same k3s flags you used.
   $k3d cluster create test --servers 1 --agents 2 --k3s-arg "--disable=servicelb@server:*" --k3s-arg "--disable=traefik@server:*" --no-lb --k3s-arg "--disable-network-policy@server:*" --k3s-arg "--flannel-backend=none@server:*"

2. Mounted ebpf file systems to the docker containers.
   $docker exec -it k3d-test-server-0 mount bpffs /sys/fs/bpf -t bpf && docker exec -it k3d-test-server-0 mount --make-shared /sys/fs/bpf
$docker exec -it k3d-test-agent-0 mount bpffs /sys/fs/bpf -t bpf && docker exec -it k3d-test-agent-0 mount --make-shared /sys/fs/bpf
$docker exec -it k3d-test-agent-1 mount bpffs /sys/fs/bpf -t bpf && docker exec -it k3d-test-agent-1 mount --make-shared /sys/fs/bpf

3. Exported the kubeconig file (I only have one cluster on my machine).
   $export KUBECONFIG=.kube/config

4. Installed Cilium with Helm.
   $helm install cilium cilium/cilium --version 1.9.1 --namespace kube-system --set kubeProxyReplacement=partial --set hostServices.enabled=false --set externalIPs.enabled=true --set nodePort.enabled=true --set hostPort.enabled=true --set bpf.masquerade=false --set image.pullPolicy=IfNotPresent --set ipam.mode=kubernetes

The installation was successful, but Cilium cannot be ready because of the Daemonsets:

K9s showing some server containers cannot be created:

Are there any steps that I have taken wrong or I have missed? Please advise and thanks in advance for your help!

[github-actions]

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

[github-actions]

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

[github-actions]

This issue has not seen any activity since it was marked stale.
Closing.