Rework to library-go

Makefile

@@ -0,0 +1,52 @@
+all: build
+.PHONY: all
+
+# Include the library makefile
+include $(addprefix ./vendor/github.com/openshift/build-machinery-go/make/, \
+	golang.mk \
+	targets/openshift/deps-gomod.mk \
+	targets/openshift/images.mk \
+	targets/openshift/bindata.mk \
+)
+
+# Run core verification and all self contained tests.
+#
+# Example:
+#   make check
+check: | verify test-unit
+.PHONY: check
+
+IMAGE_REGISTRY?=registry.svc.ci.openshift.org
+
+# This will call a macro called "build-image" which will generate image specific targets based on the parameters:
+# $0 - macro name
+# $1 - target name
+# $2 - image ref
+# $3 - Dockerfile path
+# $4 - context directory for image build
+# It will generate target "image-$(1)" for building the image and binding it as a prerequisite to target "images".
+$(call build-image,csi-driver-manila-operator,$(IMAGE_REGISTRY)/ocp/4.6:csi-driver-manila-operator,./build/Dockerfile.openshift,.)
+
+# generate bindata targets
+# $0 - macro name
+# $1 - target suffix
+# $2 - input dirs
+# $3 - prefix
+# $4 - pkg
+# $5 - output
+$(call add-bindata,generated,./assets/...,assets,generated,pkg/generated/bindata.go)
+
+clean:
+	$(RM) manila-csi-driver-operator
+.PHONY: clean
+
+GO_TEST_PACKAGES :=./pkg/... ./cmd/...
+
+# Run e2e tests. Requires openshift-tests in $PATH.
+#
+# Example:
+#   make test-e2e
+test-e2e:
+	hack/e2e.sh
+
+.PHONY: test-e2e

OWNERS

@@ -1,4 +1,5 @@
 approvers:
+  - bertinatto
   - Fedosin
   - gnufied
   - iamemilio

README.md

@@ -1,112 +1,26 @@
-# CSI Driver Manila Operator
+# Manila CSI driver operator
 
-Operator to create, configure and manage CSI driver for OpenStack Manila in OpenShift.
+An operator to deploy the [Manila CSI driver](https://github.com/openshift/cloud-provider-openstack/tree/master/pkg/csi/manila) in OpenShift.
 
-## Quick Start
+## Design
 
-### Installing the operator
+The operator is based on [openshift/library-go](https://github.com/openshift/library-go). It manages `ClusterCSIDriver` instance named `manila.csi.openstack.org` and runs several controllers in parallel:
 
-The operator needs its own namespace, service account, security context, and a few roles and bindings. For example, to install these on OpenShift >= 4.4:
+* `manilaController`: Talks to OpenStack API and checks if Manila service is provided.
+  * If Manila service is found: 
+    * It starts `manilaControllerSet`: Runs `csidriverset.Controller` that installs the Manila CSI driver itself.
+    * It starts `nfsController`: Runs `csidriverset.Controller` that installs NFS CSI driver itself.
+    * It creates `StorageClass` for each share type reported by Manila and periodically syncs them at least once per minute, in case a new share type appears in Manila.
+    * It never removes StorageClass if Manila service or share type disappears. It may be temporary OpenStack ir Manila re-configuration hiccup.
+  * If there is no Manila service, it marks the `ClusterCSIDriver` instance with `ManilaControllerDisabled: True` condition. It does not stop any CSI drivers started when Manila service was present! This allows pod to at least unmount their volumes. 
+* `secretSyncController`: Syncs Secret provided by cloud-credentials-operator into a new Secret that is used by the CSI drivers. The drivers need OpenStack credentials in different format than provided by cloud-credentials-operator.
 
-```sh
-oc apply -f deploy/namespace.yaml -f deploy/crds/csi.openshift.io_maniladrivers_crd.yaml -f deploy/service_account.yaml -f deploy/role_binding.yaml -f deploy/role.yaml -f deploy/operator.yaml
-```
+## Usage
 
-You can check logs of the operator by executing:
+Check deployment YAML files in `manifests/` directory.
 
-```sh
-oc logs -f -n openshift-manila-csi-driver-operator $(oc get pods --no-headers -n openshift-manila-csi-driver-operator -o custom-columns=":metadata.name")
-```
+The operator makes few assumptions about the namespace where it runs:
 
-### Installing the driver
-
-When the operator is started, you need to create a CR to install the driver:
-
-```sh
-oc apply -f deploy/crds/csi.openshift.io_v1alpha1_maniladriver_cr.yaml
-```
-
-**Note:** ManilaDriver CR is a singleton, which means you can't create more than one instance of this resource. By convention this should be a cluster-scoped object called `cluster`.
-
-Operator automatically creates required StorageClasses for all Manila share types. Each of them is called `manila-csi-<share_type>`.
-
-To see the list of provisioned Storage Classes execute:
-
-```sh
-oc get storageclasses
-```
-
-All driver's resources are created in  the `openshift-manila-csi-driver` namespace.
-
-### Creating PVCs and Pods
-
-You're all set now! However, you likely want to test the deployment, so let's create a PVC and POD for testing.
-
-**Note:** In the PVC example we use `manila-csi-default` Storage Class, which may be different in your case.
-
-```sh
-oc create namespace manila-test
-oc create -n manila-test -f examples/nfs/dynamic-provisioning/pvc.yaml
-```
-
-At this moment Manila CSI driver should provision a volume in the Manila service.
-
-Next step is to create a pod.
-
-```sh
-oc create -n manila-test -f examples/nfs/dynamic-provisioning/pod.yaml
-```
-
-Once the pvc and pod are up and running, it will look like this:
-
-```sh
-$ oc get pod new-nfs-share-pod
-NAME                READY   STATUS    RESTARTS   AGE
-new-nfs-share-pod   1/1     Running   0          106s
-
-$ oc get pvc -n manila-test
-NAME                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     AGE
-new-nfs-share-pvc   Bound    pvc-b1e5ebb8-8032-4722-92e3-06bd7ce5afec   1Gi        RWX            csi-manila-nfs   118s
-
-$ oc get pv
-NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                       STORAGECLASS     REASON   AGE
-pvc-b1e5ebb8-8032-4722-92e3-06bd7ce5afec   1Gi        RWX            Delete           Bound    manila-test/new-nfs-share-pvc   csi-manila-nfs            2m50s
-
-$ oc describe pod new-nfs-share-pod -n manila-test | grep Volumes: -A 4
-Volumes:
-  mypvc:
-    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
-    ClaimName:  new-nfs-share-pvc
-    ReadOnly:   false
-```
-
-Looking inside the container you will notice that the provided volume has been mounted:
-
-```sh
-$ oc exec -n manila-test -it new-nfs-share-pod  -- mount | grep /var/lib/www
-10.0.128.27:/volumes/_nogroup/e3c5f7fd-aeee-4485-9a40-6a732d55f689 on /var/lib/www type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.129.2.9,local_lock=none,addr=10.0.128.27)
-```
-
-### Delete the testing pod and pvc
-
-Eventually you want to remove all the testing resources from your cluster. To do so just delete the namespace:
-
-```sh
-oc delete namespace manila-test
-```
-
-Manila provisioner will automatically delete the share in Manila service as well.
-
-### Removing the driver and operator
-
-First, remove the CR. The driver and its cluster-scoped resources will be deleted along with it.
-
-```sh
-oc delete -f deploy/crds/csi.openshift.io_v1alpha1_maniladriver_cr.yaml
-```
-
-When the driver is deleted, remove the remaining parts of the operator.
-
-```sh
-oc delete -f deploy/crds/csi.openshift.io_maniladrivers_crd.yaml -f deploy/role.yaml -f deploy/role_binding.yaml -f deploy/service_account.yaml -f deploy/namespace.yaml
-```
+* OpenStack cloud credentials are in Secret named "cloud-credentials" in the same namespace where the operator runs. The operator uses the credentials to check if Manila is present in the cluster and, since OpenStack does not allow any fine-grained access control, it lets the CSI driver to use the same credentials.
+* If underlying OpenStack uses self-signed certificate, the operator expects the certificate is present in a ConfigMap named "cloud-provider-config" with key "ca-bundle.pem" in the namespace where it runs. Generally, it should be a copy of "openshift-config/cloud-provider-config" ConfigMap. It then uses the certificate to talk to OpenStack API.
+* The operand (= the CSI driver) must run in the same namespace as the operator, for the same reason as above - it uses the same self-signed OpenStack certificate, if provided.

assets/controller.yaml

@@ -0,0 +1,134 @@
+kind: Deployment
+apiVersion: apps/v1
+metadata:
+  name: manila-csi-driver-controller
+  namespace: openshift-cluster-csi-drivers
+spec:
+  selector:
+    matchLabels:
+      app: manila-csi-driver-controller
+  serviceName: manila-csi-driver-controller
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        app: manila-csi-driver-controller
+    spec:
+      serviceAccount: manila-csi-driver-controller-sa
+      priorityClassName: system-cluster-critical
+      tolerations:
+        - key: CriticalAddonsOnly
+          operator: Exists
+      containers:
+        - name: csi-driver
+          image: ${DRIVER_IMAGE}
+          resources:
+            requests:
+              memory: 50Mi
+              cpu: 10m
+          args:
+            - --v=${LOG_LEVEL}
+            - --nodeid=$(NODE_ID)
+            - --endpoint=$(CSI_ENDPOINT)
+            - --drivername=$(DRIVER_NAME)
+            - --share-protocol-selector=$(MANILA_SHARE_PROTO)
+            - --fwdendpoint=$(FWD_CSI_ENDPOINT)
+          env:
+            - name: DRIVER_NAME
+              value: manila.csi.openstack.org
+            - name: NODE_ID
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+            - name: CSI_ENDPOINT
+              value: unix:///plugin/csi.sock
+            - name: MANILA_SHARE_PROTO
+              value: NFS
+            - name: FWD_CSI_ENDPOINT
+              value: unix:///plugin/csi-nfs.sock
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /plugin
+            - name: cacert
+              mountPath: /usr/share/pki/ca-trust-source
+          resources:
+            requests:
+              cpu: 10m
+              memory: 50Mi
+        # TODO: fix manila CSI driver not to require NFS driver socket!
+        - name: csi-driver-nfs
+          image: ${NFS_DRIVER_IMAGE}
+          resources:
+            requests:
+              memory: 20Mi
+              cpu: 5m
+          args:
+            - "--nodeid=$(NODE_ID)"
+            - "--endpoint=unix://plugin/csi-nfs.sock"
+            - "--mount-permissions=0777"
+          env:
+            - name: NODE_ID
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /plugin
+          resources:
+            requests:
+              cpu: 10m
+              memory: 50Mi
+        - name: csi-provisioner
+          image: ${PROVISIONER_IMAGE}
+          resources:
+            requests:
+              memory: 50Mi
+              cpu: 10m
+          args:
+            - --csi-address=$(ADDRESS)
+            - --feature-gates=Topology=true
+            - --v=${LOG_LEVEL}
+            - --timeout=120s
+          env:
+            - name: ADDRESS
+              value: /var/lib/csi/sockets/pluginproxy/csi.sock
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /var/lib/csi/sockets/pluginproxy/
+          resources:
+            requests:
+              cpu: 10m
+              memory: 50Mi
+        - name: csi-snapshotter
+          image: ${SNAPSHOTTER_IMAGE}
+          resources:
+            requests:
+              memory: 50Mi
+              cpu: 10m
+          args:
+            - --csi-address=$(ADDRESS)
+            - --v=${LOG_LEVEL}
+          env:
+          - name: ADDRESS
+            value: /var/lib/csi/sockets/pluginproxy/csi.sock
+          volumeMounts:
+          - mountPath: /var/lib/csi/sockets/pluginproxy/
+            name: socket-dir
+          resources:
+            requests:
+              cpu: 10m
+              memory: 50Mi
+      volumes:
+        - name: socket-dir
+          emptyDir: {}
+        - name: cacert
+          # Extract ca-bundle.pem to /usr/share/pki/ca-trust-source if present.
+          # Let the pod start when the ConfigMap does not exist or the certificate
+          # is not preset there. The certificate file will be created once the
+          # ConfigMap is created / the cerificate is added to it.
+          configMap:
+            name: cloud-provider-config
+            items:
+            - key: ca-bundle.pem
+              path: ca-bundle.pem
+            optional: true

assets/controller_sa.yaml

@@ -0,0 +1,5 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: manila-csi-driver-controller-sa
+  namespace: openshift-cluster-csi-drivers

assets/csidriver.yaml

@@ -0,0 +1,10 @@
+apiVersion: storage.k8s.io/v1
+kind: CSIDriver
+metadata:
+  name: manila.csi.openstack.org
+  annotations:
+      # This CSIDriver is managed by an OCP CSI operator
+      csi.openshift.io/managed: "true"
+spec:
+  attachRequired: false
+  podInfoOnMount: false