CloudStack KVM + LINSTOR primary storage: VM stuck in Starting although LINSTOR volume is successfully created

[anirudh09041]

CloudStack 4.22.1 + KVM + LINSTOR: VM stuck in Starting

Hi,

I'm troubleshooting a VM deployment issue with CloudStack 4.22.1.0 using LINSTOR as primary storage.

Environment

• CloudStack: 4.22.1.0
• Hypervisor: KVM / RHEL 9.x
• LINSTOR: 1.33.1
• DRBD: 9.3.3 kernel / 9.34.0 userspace
• Zone: zone-sv2
• KVM host: sj-sv2-devop-80 (10.96.32.31)
• LINSTOR controllers: 10.96.32.193/195/197

CloudStack configuration

Primary storage:

Name: linstor-primary
Type: Linstor
Scope: ZONE
Storage tag: linstor
Path: cs-rg
Hypervisor: KVM
Status: Up


Compute offering:

Name: acos-dev-compute-offering
CPU: 16
Memory: 24576 MB
Provisioning: thin
Storage type: shared
Storage tag: linstor
Root disk size: 0 GB

Template:

base_image-0.0.7-dd_disabled_2
KVM / x86_64
QCOW2
Size: 301.20 GiB
Ready: true
Problem

VM test-vm-ak (i-2-33-VM) is created but remains stuck in Starting.

The ROOT volume is successfully created:

ROOT-33
Size: 301.20 GiB
Status: Allocated

LINSTOR also successfully creates the resource:

cs-edf8f4ea-6463-42b1-979f-4c4b49402dc0

with two replicas:

sj-sv2-devop-80   UpToDate
sj-sv2-devop-82   UpToDate

DRBD is healthy:

sj-sv2-devop-80:
  role: Secondary
  disk: UpToDate
  peer: sj-sv2-devop-82
  peer-disk: UpToDate

sj-sv2-devop-82:
  role: Secondary
  disk: UpToDate
  peer: sj-sv2-devop-80
  peer-disk: UpToDate

The DRBD device exists on the KVM host:

/dev/drbd1000
301.2G

The LINSTOR-backed libvirt storage pool is also active and reports:

Capacity:   1.75 TiB
Available:  1.70 TiB
Important observation

Despite the successful LINSTOR/DRBD provisioning, the libvirt domain is never created:

virsh list --all --name

does not show:

i-2-33-VM

and:

virsh domblklist i-2-33-VM

returns:

error: failed to get domain 'i-2-33-VM'

Interestingly, the same VM creation works successfully when NFS is used as primary storage.

I also noticed this libvirt message:

no connection driver available for lxc:///
End of file while reading data: Input/output error

although I'm not sure if it is related.

My question

Since CloudStack successfully creates the LINSTOR resource and /dev/drbd1000, but never creates the libvirt domain:

What should the expected CloudStack → LINSTOR → DRBD → libvirt workflow be at this stage, and where should I look next?

In particular:

Should CloudStack/LINSTOR promote the DRBD resource to Primary before VM creation?
How should CloudStack attach /dev/drbd1000 to libvirt/QEMU?
Is the LINSTOR libvirt storage pool required, and what should its configuration look like?
Are there any known CloudStack 4.22.1 LINSTOR/KVM issues?
Which CloudStack agent/libvirt logs would be most useful for debugging this stage?

Thanks!