feat(Azure): Incorporate azure resources

[cvvz]

Change description

feat: Incoporate azure resources

Per Azure official document, users have to complete following steps before they deploy trident and create tbc (tridentbackendconfig):

1. Create a netapp account
2. Create capacity pool
3. Create subnet and delegate to Azure NetApp Files

This PR tries to incorporate these steps into the process of initialing Azure driver in trident. Users do not need to create these Azure resources anymore. All they need is to deploy trident and then create a tbc.

After a tbc deployed, trident-orchestrator starts to initialize Azure driver, it will discover netapp account, capacity pool and subnet and they make up the virtual storage pools. What this PR implement is that when these Azure resources are not explicitly specified in tbc, and no Azure resource can be discovered, trident-orchestrator will create them.

Project tracking

Do any added TODOs have an issue in the backlog?

Did you add unit tests? Why not?

Does this code need functional testing?

Is a code review walkthrough needed? why or why not?

Should additional test coverage be executed in addition to pre-merge?

Does this code need a note in the changelog?

Yes. feat(Azure): Incorporate azure resources.

Does this code require documentation changes?

Yes.

Additional Information

Manual test steps

1. create an aks cluster

export LOCATION=""
export CLUSTER="aks-anf-demo"
export CLUSTER_GROUP="aks-anf-demo-group"
az group create -n CLUSTER_GROUP -l $LOCATION
az aks create -n $CLUSTER -g $CLUSTER_GROUP

2. add necessary permission to aks managed identity

export subID=$(az account show --query id -o tsv)
export MC_RESOURCE_GROUP=$(az aks show -n $CLUSTER -g $CLUSTER_GROUP --query "nodeResourceGroup" -o tsv)
AGENT_POOL_MSI_OBJECT_ID=`az identity show -n ${CLUSTER}-agentpool -g $MC_RESOURCE_GROUP --query "principalId" -o tsv`
AGENT_POOL_MSI_CLIENT_ID=`az identity show -n ${CLUSTER}-agentpool -g $MC_RESOURCE_GROUP --query "clientId" -o tsv`
az role assignment create --assignee $AGENT_POOL_MSI_OBJECT_ID --role Owner --scope /subscriptions/$subID/resourceGroups/$MC_RESOURCE_GROUP

3. build and push image

# trident-operator
BUILD_CLI="docker buildx" BUILDX_OUTPUT=push make operator_images

# trident
BUILD_CLI="docker buildx" PLATFORMS="linux/amd64 windows/amd64/ltsc2022 windows/amd64/1809" BUILDX_OUTPUT=push make images

# Using Docker manifest to create multi-arch images
export VERSION=$(cat ./hack/VERSION)
docker manifest create \
cvvz/trident:$VERSION-custom \
--amend cvvz/trident:$VERSION-custom-windows-amd64-ltsc2022 \
--amend cvvz/trident:$VERSION-custom-windows-amd64-1809 \
--amend cvvz/trident:$VERSION-custom-linux-amd64

docker manifest push --purge cvvz/trident:$VERSION-custom

4. deploy trident

export CP=Azure
export ENABLE_WIN=true
export VERSION=$(cat ./hack/VERSION)

helm install trident helm/trident-operator \
--create-namespace \
--namespace trident \
--set operatorImage=cvvz/trident-operator:$VERSION-custom-linux-amd64 \
--set tridentImage=cvvz/trident:$VERSION-custom \
--set imagePullPolicy=Always \
--set tridentLogLevel=trace \
--set cloudProvider=$CP \
--set windows=$ENABLE_WIN

5. create TridentBackendConfig and wait until it creates backend successfully.

# trident backend config
cat <<EOF | kubectl create -f -
apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
  name: backend-tbc-anf
  namespace: trident
spec:
  version: 1
  storageDriverName: azure-netapp-files
EOF

6. Create SC, PVC, Pod

# storage class
cat <<EOF | kubectl create -f -
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azure-netapp-files
provisioner: csi.trident.netapp.io
parameters:
  backendType: "azure-netapp-files"
  fsType: "nfs"
EOF

# PVC
cat <<EOF | kubectl create -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: anf-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: azure-netapp-files
EOF

# Pod
cat <<EOF | kubectl create -f -
kind: Pod
apiVersion: v1
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi
    volumeMounts:
    - mountPath: "/mnt/data"
      name: volume
  volumes:
    - name: volume
      persistentVolumeClaim:
        claimName: anf-pvc
EOF

[cvvz]

This PR is checkout from #829

[andyzhangx]

could you add more details in the PR description? .e.g.
currently we need to create netapp account, netapp volumes, configure vnet, what will this PR do?
https://learn.microsoft.com/en-us/azure/aks/azure-netapp-files-nfs

also there is smb volume, how to deal with that?

[andyzhangx]

let's first merge #829 first since this PR depends on that PR, and also provide a feature flag to decide whether doing following things automatically, by default it's true, and user could disable it if there is sth. wrong:

Create a netapp account
Create capacity pool
Create subnet and delegate to Azure NetApp Files

[cvvz]

I've added manual test steps in the description.

[cvvz]

I've tested with Azure official doc successfully, so there should be no regression.