Azure disk: Fail to find storage account if there is no "vhds" container in it

[weherdh]

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.1011+42d5a1a9cd4aca", GitCommit:"42d5a1a9cd4acae9c7b258c21bfc4fb4955d7be3", GitTreeState:"clean", BuildDate:"2016-11-30T07:19:12Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.1011+42d5a1a9cd4aca", GitCommit:"42d5a1a9cd4acae9c7b258c21bfc4fb4955d7be3", GitTreeState:"clean", BuildDate:"2016-11-30T07:19:12Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Cloud provider or hardware configuration: Azure
• OS (e.g. from /etc/os-release): RHEL 7.3
• Install tools: Ansible

What happened:
PVC keeps pending with a new created storage account

What you expected to happen:
Storage class can find storage account and then create a dynamic pv for pvc to bind

How to reproduce it (as minimally and precisely as possible):

1. Create a new storage account without create any container
2. Create a storage class as below

[root@wehe-master1 azure]# cat azscw-LRS.yaml 
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: wlrs 
provisioner: kubernetes.io/azure-disk
parameters:
  skuName: Standard_LRS
  location: westus 
  storageAccount: qewestus 

[root@wehe-master1 azure]# kubectl describe storageclass wlrs
Name:		wlrs
IsDefaultClass:	No
Annotations:	<none>
Provisioner:	kubernetes.io/azure-disk
Parameters:	location=westus,skuName=Standard_LRS,storageAccount=qewestus
No events.

3. Create a pvc annotating this storage class

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: azpvcwest
  annotations:        
    volume.beta.kubernetes.io/storage-class: wlrs 
spec:
  accessModes:
    - ReadWriteOnce 
  resources:
    requests:
      storage: 1Gi

4. Check pvc status is pending

[root@wehe-master1 ~]# kubectl describe pvc azpvcwest
Events:
  FirstSeen	LastSeen	Count	From				SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----				-------------	--------	------			-------
  16h		11s		3940	{persistentvolume-controller }			Warning		ProvisioningFailed	Failed to provision volume with StorageClass "wlrs": failed to find a matching storage account

Anything else do we need to know:
After I create a new container "vhds" in storage account of "qewestus", storage class works and pvc is bound

[root@wehe-master1 ~]# kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
azpvcwest   Bound     pvc-703fc37c-bc65-11e6-b2be-000d3a194418   1Gi        RWO           16h

[rootfs]

VHD container is a rerequiste. Will update the doc.

[kim0]

@rootfs .. I was about to open this exact bug. Shouldn't the provisoner create the vhds container ?

[rootfs]

yes, there is a todo for this case (also missing storage account)
https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/azure/azure_storage.go#L198

[kim0]

Another minor issue is, new volumes take a lot of time (tens of mins) to be ext4 formatted. This causes the pod to timeout and fail upon first use! Any way to make this better

[rootfs]

@kim0 fix is in #37845

[eggsy84]

Can anyone share how to create the vhds container? For those that don't yet have this fix

[jgeraerts]

@eggsy84
Easiest is through the portal, browse to your storage account and create a container called vhds

[eggsy84]

Thats great thanks for the quick response. The automator in me now wants to see if I can automate the creation of that container through powershell ;)

[rootfs]

https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-how-to-use-blobs#create-a-container

[eggsy84]

Does the container have to be of type 'Container' ? IE Full public read access??

There are three levels of anonymous read access: Off, Blob, and Container. To prevent anonymous access to blobs, set the Permission parameter to Off. By default, the new container is private and can be accessed only by the account owner. To allow anonymous public read access to blob resources, but not to container metadata or to the list of blobs in the container, set the Permission parameter to Blob. To allow full public read access to blob resources, container metadata, and the list of blobs in the container, set the Permission parameter to Container. For more information, see Manage anonymous read access to containers and blobs.