[BUG] Fail to start longhorn with k3d #206
Comments
|
Hi there, thanks for opening this issue. |
|
@iwilltry42 I have the same problem with is issues. when i try your solution in k3d
it say
It seems that it is not allowed in v3.0.0-beta.1 |
|
Hi @JinLinGan, sorry for that. Fixed it in ae9be06 |
|
Annoyingly https://cert-manager.io/docs/usage/csi/ also depends on shared mounts. But nicely so, first sight, this seemed to work $ mkdir -p /tmp/k3d/kubelet/pods
$ k3d cluster create [...] --agents 2 -v /tmp/k3d/kubelet/pods:/var/lib/kubelet/pods:shared
$ kubectl apply -f "https://github.com/jetstack/cert-manager/releases/download/v0.16.1/cert-manager.yaml"
$ kubectl apply -f "https://raw.githubusercontent.com/jetstack/cert-manager-csi/7fa27a6d05111a038fa5a21cefdcde2613f3bf4f/deploy/cert-manager-csi-driver.yaml" |
##### ISSUE TYPE
- Feature Pull Request
##### SUMMARY
- Allows providing communicator configuration via env variables
- Env variables have higher priority that config from file
- Helm chart has:
- `extraEnv`
- `extraVolumeMounts`
- `extraVolumes`
Fixes #480
Related documentation: kubeshop/botkube-docs#82
##### TESTING
Unit test proves that the reading configuration works as expected. However, below you will find an e2e tutorial.
**BotKube with Vault via CSI driver**
1. Create K8s cluster, e.g. k3s via `lima-vm`: `limactl start template://k3s`
> **NOTE:** The CSI needs to be supported, on k3d is problematic: k3d-io/k3d#206. Alternative is to just not play with the CSI driver and create your own volume that will be mounted, e.g. with predefined secret.
2. Install Vault:
```bash
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update
helm install vault hashicorp/vault \
--set "server.dev.enabled=true" \
--set "injector.enabled=false" \
--set "csi.enabled=true"
```
3. Set Slack token:
```bash
kubectl exec -it vault-0 -- /bin/sh
```
```bash
vault kv put secret/slack token={token}
```
4. Configure Kubernetes authentication:
```bash
vault auth enable kubernetes
vault write auth/kubernetes/config \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
```
```bash
vault policy write internal-app - <<EOF
path "secret/data/slack" {
capabilities = ["read"]
}
EOF
```
```bash
vault write auth/kubernetes/role/database \
bound_service_account_names=botkube-sa \
bound_service_account_namespaces=default \
policies=internal-app \
ttl=20m
```
5. Install the secrets store CSI driver:
```bash
helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm install csi secrets-store-csi-driver/secrets-store-csi-driver --set syncSecret.enabled=true
```
6. Create install parameters:
```bash
cat > /tmp/values.yaml << ENDOFFILE
extraObjects:
- apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: vault-database
spec:
provider: vault
secretObjects:
- data:
- key: token
objectName: "slack-token"
secretName: communication-slack
type: Opaque
parameters:
vaultAddress: "http://vault.default:8200"
roleName: "database"
objects: |
- objectName: "slack-token"
secretPath: "secret/data/slack"
secretKey: "token"
communications:
# Settings for Slack
slack:
enabled: true
channel: 'random'
notiftype: short
# token - specified via env
extraEnv:
- name: COMMUNICATION_SLACK_TOKEN
valueFrom:
secretKeyRef:
name: communication-slack
key: token
extraVolumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets-store"
readOnly: true
extraVolumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "vault-database"
image:
registry: mszostok
repository: botkube
tag: env-test-v2
ENDOFFILE
```
7. Checkout this PR: `gh pr checkout 601`
8. Install BotKube:
```bash
helm install botkube -f /tmp/values.yaml ./helm/botkube
```
What did you do?
I tried to deployment longhorn using
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml.It works perfectly on k3s, but failed on k3d as
Error: failed to generate container "2e4bb7f592a59476546b25d1c224c8bdb6eaeb8fb2d31709769f95cb61d7c1f5" spec: path "/var/lib/longhorn/" is mounted on "/" but it is not a shared mountHow was the cluster created?
k3d create --enable-registry --workers 2 --auto-restart --api-port 0.0.0.0:6443What did you do afterwards?
What did you expect to happen?
A successful deployment looks like this:
Screenshots or terminal output
Which OS & Architecture?
Which version of
k3d?Which version of docker?
The text was updated successfully, but these errors were encountered: