Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history
commit e26eae4
Merge: 020a6c3 141ab79
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 23 07:59:46 2023 +0000

    Merge branch 'master' of https://github.com/kubernetes-sigs/azurefile-csi-driver into support-workload-identity

commit 020a6c3
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 23 07:58:28 2023 +0000

    fix

commit e23c004
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 23 07:49:26 2023 +0000

    fix

commit 6336c4e
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 23 07:42:15 2023 +0000

    add docs

commit 7e84f91
Author: weizhichen <weizhichen@microsoft.com>
Date:   Wed Feb 22 08:15:00 2023 +0000

    fix

commit 6a866db
Author: weizhichen <weizhichen@microsoft.com>
Date:   Tue Feb 21 11:15:22 2023 +0000

    fix

commit f7caea6
Author: weizhichen <weizhichen@microsoft.com>
Date:   Tue Feb 21 08:40:31 2023 +0000

    fix

commit bccdb92
Author: weizhichen <weizhichen@microsoft.com>
Date:   Tue Feb 21 08:22:43 2023 +0000

    fix

commit 3f99c86
Author: weizhichen <weizhichen@microsoft.com>
Date:   Tue Feb 21 05:03:17 2023 +0000

    fix

commit d2663f3
Author: weizhichen <weizhichen@microsoft.com>
Date:   Tue Feb 21 04:42:51 2023 +0000

    fix

commit ca11365
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 16 10:54:38 2023 +0000

    fix

commit 0ef4233
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 16 09:59:50 2023 +0000

    support workload identity

commit b06461d
Author: weizhichen <weizhichen@microsoft.com>
Date:   Thu Feb 16 03:25:40 2023 +0000

    chore: update cloud-provider
  • Loading branch information
cvvz committed Feb 23, 2023
1 parent 141ab79 commit f0b857c
Show file tree
Hide file tree
Showing 9 changed files with 229 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,16 @@ spec:
timeoutSeconds: 10
periodSeconds: 30
env:
{{- if and .Values.workloadIdentity.clientID .Values.workloadIdentity.tenantID}}
- name: AZURE_CLIENT_ID
value: {{ .Values.workloadIdentity.clientID }}
- name: AZURE_TENANT_ID
value: {{ .Values.workloadIdentity.tenantID }}
- name: AZURE_FEDERATED_TOKEN_FILE
value: /var/run/secrets/tokens/azure-identity-token
- name: AZURE_AUTHORITY_HOST
value: https://login.microsoftonline.com/
{{- end }}
- name: AZURE_CREDENTIAL_FILE
valueFrom:
configMapKeyRef:
Expand All @@ -213,6 +223,11 @@ spec:
value: {{ .Values.driver.azureGoSDKLogLevel }}
imagePullPolicy: {{ .Values.image.azurefile.pullPolicy }}
volumeMounts:
{{- if and .Values.workloadIdentity.clientID .Values.workloadIdentity.tenantID}}
- mountPath: /var/run/secrets/tokens
name: azure-identity-token
readOnly: true
{{- end }}
- mountPath: /csi
name: socket-dir
- mountPath: /etc/kubernetes/
Expand All @@ -227,6 +242,16 @@ spec:
{{- end }}
resources: {{- toYaml .Values.controller.resources.azurefile | nindent 12 }}
volumes:
{{- if and .Values.workloadIdentity.clientID .Values.workloadIdentity.tenantID}}
- name: azure-identity-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: api://AzureADTokenExchange
expirationSeconds: 3600
path: azure-identity-token
{{- end }}
- name: socket-dir
emptyDir: {}
- name: azure-cred
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,16 @@ spec:
timeoutSeconds: 10
periodSeconds: 30
env:
{{- if and .Values.workloadIdentity.clientID .Values.workloadIdentity.tenantID}}
- name: AZURE_CLIENT_ID
value: {{ .Values.workloadIdentity.clientID }}
- name: AZURE_TENANT_ID
value: {{ .Values.workloadIdentity.tenantID }}
- name: AZURE_FEDERATED_TOKEN_FILE
value: /var/run/secrets/tokens/azure-identity-token
- name: AZURE_AUTHORITY_HOST
value: https://login.microsoftonline.com/
{{- end }}
- name: AZURE_CREDENTIAL_FILE
valueFrom:
configMapKeyRef:
Expand Down Expand Up @@ -166,6 +176,11 @@ spec:
securityContext:
privileged: true
volumeMounts:
{{- if and .Values.workloadIdentity.clientID .Values.workloadIdentity.tenantID}}
- mountPath: /var/run/secrets/tokens
name: azure-identity-token
readOnly: true
{{- end }}
- mountPath: /csi
name: socket-dir
- mountPath: {{ .Values.linux.kubelet }}/
Expand All @@ -185,6 +200,16 @@ spec:
{{- end }}
resources: {{- toYaml .Values.linux.resources.azurefile | nindent 12 }}
volumes:
{{- if and .Values.workloadIdentity.clientID .Values.workloadIdentity.tenantID}}
- name: azure-identity-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: api://AzureADTokenExchange
expirationSeconds: 3600
path: azure-identity-token
{{- end }}
- hostPath:
path: {{ .Values.linux.kubelet }}/plugins/{{ .Values.driver.name }}
type: DirectoryOrCreate
Expand Down
4 changes: 4 additions & 0 deletions charts/latest/azurefile-csi-driver/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -251,3 +251,7 @@ windows:
operator: NotIn
values:
- virtual-kubelet

workloadIdentity:
clientID: ""
tenantID: ""
139 changes: 139 additions & 0 deletions docs/workload-identity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# How to Use workload identity with Azurefile

## Prerequisites
This document is mainly refer to [Azure AD Workload Identity Quick Start](https://azure.github.io/azure-workload-identity/docs/quick-start.html). Please Complete the [Prerequisites part](https://azure.github.io/azure-workload-identity/docs/installation.html#prerequisites) of the installation guide before the following steps. Please note the [Azure AD Workload Identity Components part](https://azure.github.io/azure-workload-identity/docs/installation.html#azure-ad-workload-identity-components) is **NOT** required.

After you finish the [Prerequisites part](https://azure.github.io/azure-workload-identity/docs/installation.html#prerequisites) of the Installation guide, you should have obtained your cluster’s OIDC issuer URL.


## 1. Export environment variables
```shell
export OIDC_ISSUER="<your cluster’s OIDC issuer URL>"
export AZURE_FILE_RESOURCE_GROUP="<resource group where Azurefile storage account reside>"
export LOCATION=eastus

# environment variables for the AAD application
# [OPTIONAL] Only set this if you're using a Azure AD Application as part of this tutorial
export APPLICATION_NAME="<your application name>"

# environment variables for the user-assigned managed identity
# [OPTIONAL] Only set this if you're using a user-assigned managed identity as part of this tutorial
export USER_ASSIGNED_IDENTITY_NAME="<your user-assigned managed identity name>"
export IDENTITY_RESOURCE_GROUP="<resource group where your user-assigned managed identity reside>"

# Azurefile CSI Driver Service Account and namespace
export SA_LIST=( "csi-azurefile-controller-sa" "csi-azurefile-node-sa" )
export NAMESPACE="kube-system"
```

## 2. Create Azurefile resource group
If you are using AKS, you can get the resource group where Azurefile storage class reside by running:
```shell
export AZURE_FILE_RESOURCE_GROUP="$(az aks show --name $CLUSTER_NAME --resource-group $CLUSTER_RESOURCE_GROUP --query "nodeResourceGroup" -o tsv)"
```

You can also create resource group by yourself, but you must [specify the resource group](https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/docs/driver-parameters.md#:~:text=current%20k8s%20cluster-,resourceGroup,No,-if%20empty%2C%20driver) in the storage class while using Azurefile.
```shell
az group create -n $AZURE_FILE_RESOURCE_GROUP -l $LOCATION
```

## 3. Create an AAD application or user-assigned managed identity and grant required permissions
```shell
# create an AAD application if using Azure AD Application for this tutorial
az ad sp create-for-rbac --name "${APPLICATION_NAME}"
```

```shell
# create a user-assigned managed identity if using user-assigned managed identity for this tutorial
az identity create --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${IDENTITY_RESOURCE_GROUP}"
```

Grant required permission to the AAD application or user-assigned managed identity, for simplicity, we just assign Contributor role to the resource group where Azurefile storage class reside:

If using Azure AD Application:
```shell
export APPLICATION_CLIENT_ID="$(az ad sp list --display-name "${APPLICATION_NAME}" --query '[0].appId' -otsv)"
export AZURE_FILE_RESOURCE_GROUP_ID="$(az group show -n $AZURE_FILE_RESOURCE_GROUP --query 'id' -otsv)"
az role assignment create --assignee $APPLICATION_CLIENT_ID --role Contributor --scope $AZURE_FILE_RESOURCE_GROUP_ID
```

if using user-assigned managed identity:
```shell
export USER_ASSIGNED_IDENTITY_OBJECT_ID="$(az identity show --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${IDENTITY_RESOURCE_GROUP}" --query 'principalId' -otsv)"
export AZURE_FILE_RESOURCE_GROUP_ID="$(az group show -n $AZURE_FILE_RESOURCE_GROUP --query 'id' -otsv)"
az role assignment create --assignee $USER_ASSIGNED_IDENTITY_OBJECT_ID --role Contributor --scope $AZURE_FILE_RESOURCE_GROUP_ID
```

## 4. Establish federated identity credential between the identity and the Azurefile service account issuer & subject
If using Azure AD Application:
```shell
# Get the object ID of the AAD application
export APPLICATION_OBJECT_ID="$(az ad app show --id ${APPLICATION_CLIENT_ID} --query id -otsv)"

# Add the federated identity credential:
for SERVICE_ACCOUNT_NAME in "${SA_LIST[@]}"
do
cat <<EOF > params.json
{
"name": "${SERVICE_ACCOUNT_NAME}",
"issuer": "${OIDC_ISSUER}",
"subject": "system:serviceaccount:${NAMESPACE}:${SERVICE_ACCOUNT_NAME}",
"description": "Kubernetes service account federated credential",
"audiences": [
"api://AzureADTokenExchange"
]
}
EOF
az ad app federated-credential create --id ${APPLICATION_OBJECT_ID} --parameters @params.json
done
```

If using user-assigned managed identity:
```shell
for SERVICE_ACCOUNT_NAME in "${SA_LIST[@]}"
do
az identity federated-credential create \
--name "${SERVICE_ACCOUNT_NAME}" \
--identity-name "${USER_ASSIGNED_IDENTITY_NAME}" \
--resource-group "${IDENTITY_RESOURCE_GROUP}" \
--issuer "${OIDC_ISSUER}" \
--subject system:serviceaccount:"${NAMESPACE}":"${SERVICE_ACCOUNT_NAME}"
done
```

## 5. Deploy Azurefile

Deploy storageclass:
```shell
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azurefile-csi-driver/master/deploy/example/storageclass-azurefile-csi.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azurefile-csi-driver/master/deploy/example/storageclass-azurefile-nfs.yaml
```

Deploy Azurefile(If you are using AKS, please disable the managed Azurefile CSI driver by `--disable-file-driver` first)

If using Azure AD Application:
```shell
export CLIENT_ID="$(az ad sp list --display-name "${APPLICATION_NAME}" --query '[0].appId' -otsv)"
export TENANT_ID="$(az ad sp list --display-name "${APPLICATION_NAME}" --query '[0].appOwnerOrganizationId' -otsv)"
helm install azurefile-csi-driver charts/latest/azurefile-csi-driver \
--namespace $NAMESPACE \
--set workloadIdentity.clientID=$CLIENT_ID
--set workloadIdentity.tenantID=$TENANT_ID
```

If using user-assigned managed identity:
```shell
export CLIENT_ID="$(az identity show --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${IDENTITY_RESOURCE_GROUP}" --query 'clientId' -otsv)"
export TENANT_ID="$(az identity show --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${IDENTITY_RESOURCE_GROUP}" --query 'tenantId' -otsv)"
helm install azurefile-csi-driver charts/latest/azurefile-csi-driver \
--namespace $NAMESPACE \
--set workloadIdentity.clientID=$CLIENT_ID
--set workloadIdentity.tenantID=$TENANT_ID
```

## 6. Deploy application using Azurefile
```shell
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azurefile-csi-driver/master/deploy/example/nfs/statefulset.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azurefile-csi-driver/master/deploy/example/deployment.yaml
```
Please make sure all the Pods are running.
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -165,4 +165,5 @@ replace (
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.26.0
k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.26.0
k8s.io/sample-controller => k8s.io/sample-controller v0.26.0
sigs.k8s.io/cloud-provider-azure => ../cloud-provider-azure
)
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -883,8 +883,6 @@ rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.35 h1:+xBL5uTc+BkPBwmMi3vYfUJjq+N3K+H6PXeETwf5cPI=
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.35/go.mod h1:WxjusMwXlKzfAs4p9km6XJRndVt2FROgMVCE4cdohFo=
sigs.k8s.io/cloud-provider-azure v1.26.1-0.20230219211137-7a494eba9870 h1:8Q8TaoDKDRfeyOhHWnVv51KV2rOGAZzOfB0APl9A5qY=
sigs.k8s.io/cloud-provider-azure v1.26.1-0.20230219211137-7a494eba9870/go.mod h1:x0zqcw757LNpJ02XgDhYFV/HgkOy2C1VKhQt9UnzPTA=
sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k=
sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE=
Expand Down
3 changes: 2 additions & 1 deletion vendor/modules.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1229,7 +1229,7 @@ sigs.k8s.io/apiserver-network-proxy/konnectivity-client/pkg/client
sigs.k8s.io/apiserver-network-proxy/konnectivity-client/pkg/client/metrics
sigs.k8s.io/apiserver-network-proxy/konnectivity-client/pkg/common/metrics
sigs.k8s.io/apiserver-network-proxy/konnectivity-client/proto/client
# sigs.k8s.io/cloud-provider-azure v1.26.1-0.20230219211137-7a494eba9870
# sigs.k8s.io/cloud-provider-azure v1.26.1-0.20230219211137-7a494eba9870 => ../cloud-provider-azure
## explicit; go 1.19
sigs.k8s.io/cloud-provider-azure/pkg/azureclients
sigs.k8s.io/cloud-provider-azure/pkg/azureclients/armclient
Expand Down Expand Up @@ -1327,3 +1327,4 @@ sigs.k8s.io/yaml
# k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.26.0
# k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.26.0
# k8s.io/sample-controller => k8s.io/sample-controller v0.26.0
# sigs.k8s.io/cloud-provider-azure => ../cloud-provider-azure
12 changes: 12 additions & 0 deletions vendor/sigs.k8s.io/cloud-provider-azure/pkg/provider/azure.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit f0b857c

Please sign in to comment.