Changes in volumes directory's underlying filesystem are not reflected in driver's state at runtime

[rzetelskik]

If you run the provisioner with volumes-dir pointing to a directory with preexisting volume state files, it will account for them when building the state, which will also be reflected in CSIStorageCapacity.

$ ls -ls --block-size=K /mnt/persistent-volumes
total 4K
0K drwxr-x--- 6 root root 1K Jul  7 15:16 347e0693-f832-4fc1-a2dc-4928843b3238
4K -rw-r--r-- 1 root root 1K Jul  7 15:15 347e0693-f832-4fc1-a2dc-4928843b3238.json

$ df -h -BK /mnt/persistent-volumes/
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/loop5     20961280K 343484K 20617796K   2% /mnt/persistent-volumes

$ kubectl get csistoragecapacities.storage.k8s.io  csisc-6pnmh -ogo-template --template '{{ .capacity }}'
15718392Ki%

Yet if you delete the volume state file from the volumes directory, the change won't be reflected in the CSIStorageCapacity:

$ ls -ls --block-size=K /mnt/persistent-volumes
total 0K

$ df -h -BK /mnt/persistent-volumes/
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/loop5     20961280K 179204K 20782076K   1% /mnt/persistent-volumes

$ kubectl get csistoragecapacities.storage.k8s.io  csisc-6pnmh -ogo-template --template '{{ .capacity }}'
15718392Ki%

This can lead to the driver being unable to provision PVs despite having enough available storage.

The expected behaviour is for the driver to update capacity on changes in the filesystem of its volumes directory.

cc @zimnx