Installing network-manager-l2tp breaks microk8s

[turowicz]

We have a need of having all of our kubernetes nodes that are deployed to our clients to be available on our VPN. Unfortunately the following results in a broken node/cluster:

sudo microk8s enable host-access
sudo apt install network-manager network-manager-l2tp

After this we instantly see all pods coming down with the following network error:

coredns example:

E0628 11:46:11.700296 1 reflector.go:125] pkg/mod/k8s.io/client-go@v0.0.0-20190620085101-78d2af792bab/tools/cache/reflector.go:98: Failed to list *v1.Endpoints: Get https://10.152.183.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.152.183.1:443: connect: no route to host

calico-kube-controllers example:

21-06-28 11:46:54.860 [ERROR][1] main.go 203: Failed to verify datastore error=Get https://10.152.183.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default: context deadline exceeded

Is there any way of having a debian-based machine with Microk8s and L2TP?

[turowicz]

Sometimes it breaks even without enabling host-access.

[turowicz]

cc @ktsakalozos

[turowicz]

This is happening even with network-manager not managing any interfaces.

[turowicz]

The only way to recover the cluster is:

sudo apt purge network-manager network-manager-l2tp
sudo apt autoremove

The autoremove actually makes the node recover. Meaning there must be a dependency of network-manager-l2tp that is the culprit.

[turowicz]

One of the following dependencies break microk8s:

~$ sudo apt autoremove
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  libcharon-extauth-plugins libstrongswan libstrongswan-standard-plugins strongswan strongswan-charon strongswan-libcharon strongswan-starter xl2tpd
0 upgraded, 0 newly installed, 8 to remove and 0 not upgraded.
After this operation, 4 399 kB disk space will be freed.
Do you want to continue? [Y/n] 

[turowicz]

strongswan is the dependency of l2tp that breaks the cluster

[turowicz]

This breaks calico basically.

[turowicz]

cc @balchua @matlad @joedborg

[turowicz]

cc @ktsakalozos

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.