[BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s

[github-actions]

backport #5693

[chriscchien]

Verified pass on longhorn v1.5.x e0aafa

Steps

1. Install k3d curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
2. Create k3s cluster with custom image

k3d cluster create prod \
  --agents 2 \
  --agents-memory 16g \
  --servers 1 \
  --servers-memory 4g \
  -p "80:80@loadbalancer" \
  -p "443:443@loadbalancer" \
  -v /mnt/longhorn:/var/lib/longhorn:shared@all \
  -i chanow/k3d:iscsi

3. Exec into each k3d container perform command mount --make-shared /
4. Deploy Longhorn v1.5.x, all pod running as below

/ # k get engineimage -A
NAMESPACE         NAME          STATE      IMAGE                                    REFCOUNT   BUILDDATE   AGE
longhorn-system   ei-a786fad2   deployed   longhornio/longhorn-engine:v1.5.x-head   0          13d         60s
/ # 
/ # k get nodes
NAME                STATUS   ROLES                  AGE   VERSION
k3d-prod-agent-1    Ready    <none>                 17m   v1.29.2+k3s1
k3d-prod-server-0   Ready    control-plane,master   18m   v1.29.2+k3s1
k3d-prod-agent-0    Ready    <none>                 18m   v1.29.2+k3s1
/ # 
/ # k get pods -A
NAMESPACE         NAME                                                READY   STATUS      RESTARTS   AGE
kube-system       coredns-6799fbcd5-zxmlb                             1/1     Running     0          17m
kube-system       local-path-provisioner-6c86858495-f4g2w             1/1     Running     0          17m
kube-system       helm-install-traefik-crd-knmjr                      0/1     Completed   0          17m
kube-system       svclb-traefik-6a89d872-2bvkx                        2/2     Running     0          17m
kube-system       svclb-traefik-6a89d872-5ttfv                        2/2     Running     0          17m
kube-system       svclb-traefik-6a89d872-dcqm5                        2/2     Running     0          17m
kube-system       helm-install-traefik-vpw2v                          0/1     Completed   1          17m
kube-system       traefik-f4564c4f4-wk8v2                             1/1     Running     0          17m
kube-system       metrics-server-67c658944b-5cd6z                     1/1     Running     0          17m
longhorn-system   longhorn-ui-686d899674-29vc4                        1/1     Running     0          72s
longhorn-system   longhorn-manager-ngr8t                              1/1     Running     0          72s
longhorn-system   longhorn-ui-686d899674-7lc5t                        1/1     Running     0          72s
longhorn-system   longhorn-manager-5tnrm                              1/1     Running     0          72s
longhorn-system   longhorn-manager-ctmhs                              1/1     Running     0          72s
longhorn-system   instance-manager-2db693df321176ab7e635a179ba99bc8   1/1     Running     0          67s
longhorn-system   instance-manager-a8b100f44214f2482b028b89ff8afb54   1/1     Running     0          67s
longhorn-system   instance-manager-f9c879fe8a8980e6f200951b1eb6d18b   1/1     Running     0          65s
longhorn-system   engine-image-ei-a786fad2-scljx                      1/1     Running     0          67s
longhorn-system   engine-image-ei-a786fad2-lpxq8                      1/1     Running     0          67s
longhorn-system   engine-image-ei-a786fad2-qhxsc                      1/1     Running     0          67s
longhorn-system   longhorn-driver-deployer-c8cbb6458-7d7h9            1/1     Running     0          72s
longhorn-system   csi-attacher-578d775689-7x6wc                       1/1     Running     0          51s
longhorn-system   csi-attacher-578d775689-w6wmp                       1/1     Running     0          51s
longhorn-system   csi-attacher-578d775689-l2cg6                       1/1     Running     0          51s
longhorn-system   csi-snapshotter-7f8c7645b-6mrw2                     1/1     Running     0          51s
longhorn-system   csi-resizer-68c7997d6b-29wqt                        1/1     Running     0          51s
longhorn-system   csi-resizer-68c7997d6b-cscxv                        1/1     Running     0          51s
longhorn-system   csi-snapshotter-7f8c7645b-k4xr5                     1/1     Running     0          51s
longhorn-system   csi-provisioner-6bbd887cf9-j4q64                    1/1     Running     0          51s
longhorn-system   csi-resizer-68c7997d6b-hd4xv                        1/1     Running     0          51s
longhorn-system   csi-snapshotter-7f8c7645b-szg4q                     1/1     Running     0          51s
longhorn-system   longhorn-csi-plugin-8vj8d                           3/3     Running     0          51s
longhorn-system   csi-provisioner-6bbd887cf9-58ll9                    1/1     Running     0          51s
longhorn-system   longhorn-csi-plugin-4lsg2                           3/3     Running     0          51s
longhorn-system   csi-provisioner-6bbd887cf9-qqd9f                    1/1     Running     0          51s
longhorn-system   longhorn-csi-plugin-j7wrl                           3/3     Running     0          51s