waiting for a volume to be created, either by external provisioner "ceph.rook.io/block"

[rgstephens]

I'm new to Rook & Ceph and I followed the quickstart guide. The ceph status commands look good and I have the dashboard up and it looks good.

When I define a PVC the status stays Pending and I get the error message waiting for a volume to be created, either by external provisioner "ceph.rook.io/block".

Here's an example PVC:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-ceph-pvc2
  namespace: monitoring
spec:
  storageClassName: rook-ceph-block
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1M

I've looked at the csi-rbdplugin-provsioner... pods and see a message CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments. I also tried to run the rbd status command but I'm not clear what container I should be testing this in.

[travisn]

@rgstephens Try this troubleshooting guide for pvcs staying in pending state. For example, do you have OSDs created?

[rgstephens]

Thanks for the link, ceph status:

  cluster:
    id:     2ac66146-20e0-4381-bd5d-6d37f37eff8a
    health: HEALTH_OK
 
  services:
    mon: 3 daemons, quorum a,b,c (age 13d)
    mgr: a(active, since 2w)
    osd: 3 osds: 3 up (since 13d), 3 in (since 13d)
 
  data:
    pools:   2 pools, 33 pgs
    objects: 1 objects, 0 B
    usage:   3.0 GiB used, 5.9 TiB / 5.9 TiB avail
    pgs:     33 active+clean

The provisioner logs, csi-rbdplugin-provisioner-*, startup and one of them acquires the leader lease but they show no activity when I create a PVC which led me to the CSI Troubleshooting steps.

There's a provisioner startup message mentioned above: CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments.

Who's supposed to be making the call to the provisioner as part of the PVC create process? Maybe there is a log message in that pod about a connectivity issue.

[rgstephens]

I found an issue in the rook-operator which lead me to this known rook issue. I needed to have the rook-discover enabled. This should be enabled by default. I set ROOK_ENABLE_DISCOVERY_DAEMON: "true".

The ceph orch device ls command did not show any devices until I enabled discovery.

ceph mgr module enable rook
ceph orch set backend rook
ceph orch device ls

But I'm still getting the event from the persistentvolume-controller

waiting for a volume to be created, either by external provisioner "ceph.rook.io/block" or manually created by system administrator

[Madhu-1]

waiting for a volume to be created, either by external provisioner "ceph.rook.io/block"

Looks like you are using the Flex volume. can you please try with the CSI driver? Flex driver is disabled by default in Rook. what version of Rook you are using?