Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot install cstor on Microk8s #394

Open
RafaelMoreira1180778 opened this issue Sep 24, 2021 · 2 comments
Open

Cannot install cstor on Microk8s #394

RafaelMoreira1180778 opened this issue Sep 24, 2021 · 2 comments
Labels
openforce project/community

Comments

@RafaelMoreira1180778
Copy link

RafaelMoreira1180778 commented Sep 24, 2021 •
edited
Loading

Microk8s information:

microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    dashboard            # The Kubernetes dashboard
    dns                  # CoreDNS
    ha-cluster           # Configure high availability on the current node
    helm                 # Helm 2 - the package manager for Kubernetes
    helm3                # Helm 3 - Kubernetes package manager
    ingress              # Ingress controller for external access
    metrics-server       # K8s Metrics Server for API access to service metrics
    portainer            # Portainer UI for your Kubernetes cluster
    storage              # Storage class; allocates storage from host directory
    openebs              # OpenEBS is the open-source storage solution for Kubernetes
  disabled:
    ambassador           # Ambassador API Gateway and Ingress
    cilium               # SDN, fast with full network policy
    fluentd              # Elasticsearch-Fluentd-Kibana logging and monitoring
    gpu                  # Automatic enablement of Nvidia CUDA
    host-access          # Allow Pods connecting to Host services smoothly
    istio                # Core Istio service mesh services
    jaeger               # Kubernetes Jaeger operator with its simple config
    keda                 # Kubernetes-based Event Driven Autoscaling
    knative              # The Knative framework on Kubernetes.
    kubeflow             # Kubeflow for easy ML deployments
    linkerd              # Linkerd is a service mesh for Kubernetes and other frameworks
    metallb              # Loadbalancer for your Kubernetes cluster
    multus               # Multus CNI enables attaching multiple network interfaces to pods
    openfaas             # openfaas serverless framework
    prometheus           # Prometheus operator for monitoring and logging
    rbac                 # Role-Based Access Control for authorisation
    registry             # Private image registry exposed on localhost:32000
    traefik              # traefik Ingress controller for external access

The command I am running:

microk8s helm3 install openebs-cstor openebs-cstor/cstor -n openebs --set-string csiNode.kubeletDir="/var/snap/microk8s/common/var/lib/kubelet/" --set openebsNDM.enabled=false

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /var/snap/microk8s/2487/credentials/client.config
Error: failed to download "openebs-cstor/cstor" (hint: running `helm repo update` may help)

Is this an error on my side? I am following this documentation.
@RafaelMoreira1180778
Copy link
Author

I just tried to install using Operator and git this output:

microk8s kubectl apply -f https://openebs.github.io/charts/microk8s-cstor-operator.yaml

namespace/openebs unchanged
serviceaccount/openebs-cstor-operator unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-operator unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-operator unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-migration unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-migration unchanged
customresourcedefinition.apiextensions.k8s.io/cstorbackups.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorcompletedbackups.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorpoolclusters.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorpoolinstances.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorrestores.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumeconfigs.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumepolicies.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumereplicas.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumes.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/upgradetasks.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/migrationtasks.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumeattachments.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io configured
csidriver.storage.k8s.io/cstor.csi.openebs.io unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-snapshotter-binding unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-snapshotter-role unchanged
serviceaccount/openebs-cstor-csi-controller-sa unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-provisioner-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-provisioner-binding unchanged
priorityclass.scheduling.k8s.io/openebs-cstor-csi-controller-critical configured
priorityclass.scheduling.k8s.io/openebs-cstor-csi-node-critical configured
statefulset.apps/openebs-cstor-csi-controller unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-attacher-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-attacher-binding unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-cluster-registrar-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-cluster-registrar-binding unchanged
serviceaccount/openebs-cstor-csi-node-sa unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-registrar-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-registrar-binding unchanged
configmap/openebs-cstor-csi-iscsiadm unchanged
daemonset.apps/openebs-cstor-csi-node unchanged
deployment.apps/cspc-operator unchanged
deployment.apps/cvc-operator unchanged
service/cvc-operator-service unchanged
deployment.apps/openebs-cstor-admission-server configured
customresourcedefinition.apiextensions.k8s.io/blockdevices.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/blockdeviceclaims.openebs.io configured
configmap/openebs-ndm-config unchanged
Warning: resource daemonsets/openebs-ndm is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
Warning: resource deployments/openebs-ndm-operator is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/openebs-ndm-cluster-exporter unchanged
service/openebs-ndm-cluster-exporter-service unchanged
daemonset.apps/openebs-ndm-node-exporter unchanged
service/openebs-ndm-node-exporter-service unchanged
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"name\":\"openebs-ndm\",\"openebs.io/component-name\":\"ndm\",\"openebs.io/version\":\"3.0.0\"},\"name\":\"openebs-ndm\",\"namespace\":\"openebs\"},\"spec\":{\"selector\":{\"matchLabels\":{\"name\":\"openebs-ndm\",\"openebs.io/component-name\":\"ndm\"}},\"template\":{\"metadata\":{\"labels\":{\"name\":\"openebs-ndm\",\"openebs.io/component-name\":\"ndm\",\"openebs.io/version\":\"3.0.0\"}},\"spec\":{\"containers\":[{\"args\":[\"-v=4\",\"--feature-gates=\\\"GPTBasedUUID\\\"\"],\"env\":[{\"name\":\"NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}},{\"name\":\"NODE_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"spec.nodeName\"}}},{\"name\":\"SPARSE_FILE_DIR\",\"value\":\"/var/openebs/sparse\"},{\"name\":\"SPARSE_FILE_SIZE\",\"value\":\"10737418240\"},{\"name\":\"SPARSE_FILE_COUNT\",\"value\":\"0\"}],\"image\":\"openebs/node-disk-manager:1.7.0\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"exec\":{\"command\":[\"pgrep\",\"ndm\"]},\"initialDelaySeconds\":30,\"periodSeconds\":60},\"name\":\"node-disk-manager\",\"securityContext\":{\"privileged\":true},\"volumeMounts\":[{\"mountPath\":\"/host/node-disk-manager.config\",\"name\":\"config\",\"readOnly\":true,\"subPath\":\"node-disk-manager.config\"},{\"mountPath\":\"/run/udev\",\"name\":\"udev\"},{\"mountPath\":\"/host/proc\",\"name\":\"procmount\",\"readOnly\":true},{\"mountPath\":\"/dev\",\"name\":\"devmount\"},{\"mountPath\":\"/var/openebs/ndm\",\"name\":\"basepath\"},{\"mountPath\":\"/var/openebs/sparse\",\"name\":\"sparsepath\"}]}],\"hostNetwork\":true,\"serviceAccountName\":\"openebs-cstor-operator\",\"volumes\":[{\"configMap\":{\"name\":\"openebs-ndm-config\"},\"name\":\"config\"},{\"hostPath\":{\"path\":\"/run/udev\",\"type\":\"Directory\"},\"name\":\"udev\"},{\"hostPath\":{\"path\":\"/proc\",\"type\":\"Directory\"},\"name\":\"procmount\"},{\"hostPath\":{\"path\":\"/dev\",\"type\":\"Directory\"},\"name\":\"devmount\"},{\"hostPath\":{\"path\":\"/var/openebs/ndm\",\"type\":\"DirectoryOrCreate\"},\"name\":\"basepath\"},{\"hostPath\":{\"path\":\"/var/openebs/sparse\"},\"name\":\"sparsepath\"}]}},\"updateStrategy\":{\"type\":\"RollingUpdate\"}}}\n"},"labels":{"name":"openebs-ndm"}},"spec":{"selector":{"matchLabels":{"name":"openebs-ndm","openebs.io/component-name":"ndm"}},"template":{"spec":{"$setElementOrder/containers":[{"name":"node-disk-manager"}],"$setElementOrder/volumes":[{"name":"config"},{"name":"udev"},{"name":"procmount"},{"name":"devmount"},{"name":"basepath"},{"name":"sparsepath"}],"containers":[{"args":["-v=4","--feature-gates=\"GPTBasedUUID\""],"env":[{"name":"NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"NODE_NAME","valueFrom":{"fieldRef":{"fieldPath":"spec.nodeName"}}},{"name":"SPARSE_FILE_DIR","value":"/var/openebs/sparse"},{"name":"SPARSE_FILE_SIZE","value":"10737418240"},{"name":"SPARSE_FILE_COUNT","value":"0"}],"image":"openebs/node-disk-manager:1.7.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["pgrep","ndm"]},"initialDelaySeconds":30,"periodSeconds":60},"name":"node-disk-manager","securityContext":{"privileged":true},"volumeMounts":[{"mountPath":"/host/node-disk-manager.config","name":"config","readOnly":true,"subPath":"node-disk-manager.config"},{"mountPath":"/run/udev","name":"udev"},{"mountPath":"/host/proc","name":"procmount","readOnly":true},{"mountPath":"/dev","name":"devmount"},{"mountPath":"/var/openebs/ndm","name":"basepath"},{"mountPath":"/var/openebs/sparse","name":"sparsepath"}]}],"serviceAccountName":"openebs-cstor-operator","volumes":[{"$retainKeys":["hostPath","name"],"hostPath":{"path":"/var/openebs/ndm"},"name":"basepath"},{"$retainKeys":["hostPath","name"],"hostPath":{"path":"/var/openebs/sparse"},"name":"sparsepath"}]}}}}
to:
Resource: "apps/v1, Resource=daemonsets", GroupVersionKind: "apps/v1, Kind=DaemonSet"
Name: "openebs-ndm", Namespace: "openebs"
for: "https://openebs.github.io/charts/microk8s-cstor-operator.yaml": DaemonSet.apps "openebs-ndm" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "component":"ndm", "name":"openebs-ndm", "openebs.io/component-name":"ndm", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"name\":\"openebs-ndm-operator\",\"openebs.io/component-name\":\"ndm-operator\",\"openebs.io/version\":\"3.0.0\"},\"name\":\"openebs-ndm-operator\",\"namespace\":\"openebs\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"name\":\"openebs-ndm-operator\",\"openebs.io/component-name\":\"ndm-operator\"}},\"strategy\":{\"type\":\"Recreate\"},\"template\":{\"metadata\":{\"labels\":{\"name\":\"openebs-ndm-operator\",\"openebs.io/component-name\":\"ndm-operator\",\"openebs.io/version\":\"3.0.0\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"WATCH_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}},{\"name\":\"POD_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.name\"}}},{\"name\":\"SERVICE_ACCOUNT\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"spec.serviceAccountName\"}}},{\"name\":\"OPERATOR_NAME\",\"value\":\"node-disk-operator\"},{\"name\":\"CLEANUP_JOB_IMAGE\",\"value\":\"openebs/linux-utils:3.0.0\"}],\"image\":\"openebs/node-disk-operator:1.7.0\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"httpGet\":{\"path\":\"/healthz\",\"port\":8585},\"initialDelaySeconds\":15,\"periodSeconds\":20},\"name\":\"node-disk-operator\",\"readinessProbe\":{\"httpGet\":{\"path\":\"/readyz\",\"port\":8585},\"initialDelaySeconds\":5,\"periodSeconds\":10}}],\"serviceAccountName\":\"openebs-cstor-operator\"}}}}\n"},"labels":{"name":"openebs-ndm-operator"}},"spec":{"selector":{"matchLabels":{"name":"openebs-ndm-operator","openebs.io/component-name":"ndm-operator"}},"template":{"metadata":{"labels":{"name":"openebs-ndm-operator"}},"spec":{"$setElementOrder/containers":[{"name":"node-disk-operator"}],"containers":[{"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"SERVICE_ACCOUNT","valueFrom":{"fieldRef":{"fieldPath":"spec.serviceAccountName"}}},{"name":"OPERATOR_NAME","value":"node-disk-operator"},{"name":"CLEANUP_JOB_IMAGE","value":"openebs/linux-utils:3.0.0"}],"image":"openebs/node-disk-operator:1.7.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"path":"/healthz","port":8585},"initialDelaySeconds":15,"periodSeconds":20},"name":"node-disk-operator","readinessProbe":{"httpGet":{"path":"/readyz","port":8585},"initialDelaySeconds":5,"periodSeconds":10}}],"serviceAccountName":"openebs-cstor-operator"}}}}
to:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "openebs-ndm-operator", Namespace: "openebs"
for: "https://openebs.github.io/charts/microk8s-cstor-operator.yaml": Deployment.apps "openebs-ndm-operator" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "name":"openebs-ndm-operator", "openebs.io/component-name":"ndm-operator", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

@prateekpandey14
Copy link
Contributor

@RafaelMoreira1180778 did you already installed the openebs operator ? looks like you already have NDM components running in cluster, and you have to skip the installation of NDM deployments in cstor-operator.yaml file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
openforce project/community
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@prateekpandey14 @mittachaitu @RafaelMoreira1180778 @giridharshankar013