Support VDDK imports into RBD storage

[andrijapanicsb]

Companion main/4.23 PR: #13270

This PR is intended to keep the 4.22 and main branches aligned for the VDDK-to-RBD import feature.

Description

This PR extends the existing VMware-to-KVM VDDK import flow with Ceph RBD support, while preserving the current VDDK/QCOW2 behaviour.

The existing VDDK/QCOW2 writer remains supported in both existing modes:

• With forceconverttopool=true, CloudStack continues to let virt-v2v write QCOW2 output directly into the selected supported primary storage pool using -o local -os <pool-path> -of qcow2.
• With forceconverttopool=false, CloudStack continues to use the staged flow: virt-v2v writes finalized QCOW2 disks to temporary conversion storage, and the import step then moves/copies those disks to the final destination pool.

This change adds a destination-aware writer model for VDDK imports. The current file/QCOW2 writer path remains, and a new Ceph RBD/raw writer path is added. This gives the VDDK import flow a cleaner foundation for adding more destination-specific writers later.

For Ceph RBD, two modes are supported:

1. Direct RBD mode

   When usevddk=true, forceconverttopool=true, and convertinstancepoolid points to an RBD primary storage pool, CloudStack can write the VMware disk data directly into raw RBD images.

   This path does not use the normal virt-v2v -o local -of qcow2 writer, because RBD is not a local filesystem target. Instead, the conversion host:

   • exposes the VMware source disks through VDDK/nbdkit
   • writes the data directly into raw RBD images using qemu-img convert
   • runs guest finalization in place on the newly-created RBD images using virt-v2v-in-place, or virt-v2v --in-place when supported

   The in-place finalization happens only on the destination RBD images. The VMware source disks are not modified.

2. Staged RBD mode

   When usevddk=true and forceconverttopool=false, RBD can still be used as the final destination through the existing staged model.

   In this mode:

   • regular VDDK/virt-v2v creates finalized QCOW2 disks on temporary conversion storage
   • the import step then converts/copies those QCOW2 disks into raw RBD images

   This mode does not require in-place virt-v2v, because regular virt-v2v has already finalized the guest while creating the temporary QCOW2 disks.

Direct RBD mode requires newer conversion-host tooling. The practical recommended baseline is EL9-family KVM hosts, such as Oracle Linux 9, Rocky Linux 9, AlmaLinux 9, or RHEL 9, and Ubuntu 24.04-style hosts where virt-v2v-in-place or virt-v2v --in-place is available. EL8 and Ubuntu 22.04-style hosts should use staged RBD import unless compatible in-place tooling has been explicitly installed and detected by CloudStack.

The implementation adds KVM host capability detection for:

• VDDK RBD direct import support
• in-place virt-v2v support
• qemu-img RBD support
• qemu-based RBD copy support

Direct RBD mode is rejected early when the selected conversion host does not support the required in-place finalization path. Staged RBD remains available in that case.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

N/A

How Has This Been Tested?

Focused compile and unit test coverage was run for the changed API/core/server/KVM paths, including:

git diff --check
mvn -pl api,core -am -DskipTests -Dcheckstyle.skip=true compile
mvn -pl engine/orchestration -am -DskipTests -Dcheckstyle.skip=true compile
mvn -pl plugins/hypervisors/kvm -am -DskipTests -Dcheckstyle.skip=true compile
mvn -pl plugins/hypervisors/kvm -am -Dtest=LibvirtCheckConvertInstanceCommandWrapperTest,LibvirtConvertInstanceCommandWrapperTest,LibvirtImportConvertedInstanceCommandWrapperTest -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false test
mvn -pl server -am -Dtest=UnmanagedVMsManagerImplTest#testValidateSelectedConversionStoragePoolForVddkSkipsTemporaryPoolForStagedImport+testSelectKVMHostForConversionInClusterDirectRbdAutoSelectsHostWithDirectSupport+testSelectKVMHostForConversionInClusterDirectRbdFailsWithoutDirectSupport+testCheckConversionStoragePoolRbdAllowedForVddkForceConvertToPool -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false test

The tests cover direct RBD host capability checks, rejection when direct RBD support is missing, staged RBD destination handling, RBD destination pool type propagation, RBD disk metadata, and server-side validation for RBD storage pools.

How did you try to break this feature and the system with this change?

Negative tests were added for hosts without direct VDDK/RBD support, and staged RBD import is validated separately so that lack of in-place virt-v2v blocks only direct RBD mode, not RBD import as a whole. Existing VDDK/QCOW2 direct and staged flows are kept on the existing writer path unless the selected forced conversion pool is RBD.

[codecov]

Codecov Report

❌ Patch coverage is 26.08696% with 340 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.68%. Comparing base (21b2025) to head (46ad18e).

Files with missing lines	Patch %	Lines
.../wrapper/LibvirtConvertInstanceCommandWrapper.java	4.02%	187 Missing and 4 partials ⚠️
.../apache/cloudstack/vm/UnmanagedVMsManagerImpl.java	59.04%	37 Missing and 6 partials ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java	0.00%	25 Missing ⚠️
...com/cloud/agent/api/to/VmwareVddkSourceDiskTO.java	0.00%	21 Missing ⚠️
...java/com/cloud/agent/manager/AgentManagerImpl.java	0.00%	18 Missing ⚠️
.../LibvirtImportConvertedInstanceCommandWrapper.java	72.54%	9 Missing and 5 partials ⚠️
.../java/com/cloud/agent/api/to/RemoteInstanceTO.java	0.00%	6 Missing ⚠️
.../org/apache/cloudstack/vm/UnmanagedInstanceTO.java	0.00%	6 Missing ⚠️
...va/com/cloud/agent/api/ConvertInstanceCommand.java	0.00%	6 Missing ⚠️
...loud/agent/api/ImportConvertedInstanceCommand.java	16.66%	5 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##               4.22   #13294    +/-   ##
==========================================
  Coverage     17.67%   17.68%            
- Complexity    15792    15827    +35     
==========================================
  Files          5922     5923     +1     
  Lines        533165   533595   +430     
  Branches      65208    65259    +51     
==========================================
+ Hits          94242    94361   +119     
- Misses       428276   428583   +307     
- Partials      10647    10651     +4     

Flag	Coverage Δ
uitests	3.69% <ø> (ø)
unittests	18.75% <26.08%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[andrijapanicsb]

@blueorangutan package kvm

[blueorangutan]

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18103

[andrijapanicsb]

@blueorangutan help

[blueorangutan]

@andrijapanicsb [SL] I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv] [qemuEv] [basicZone|securityGroups]
Mgmt OS options: ['suse15', 'alma10', 'ol10', 'rocky10', 'alma9', 'centos7', 'centos6', 'rocky9', 'alma8', 'ubuntu18', 'ol9', 'ol8', 'ubuntu22', 'debian12', 'ubuntu20', 'rocky8', 'ubuntu24']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-rocky9', 'kvm-rocky10', 'kvm-ol8', 'kvm-ol9', 'kvm-ol10', 'kvm-alma8', 'kvm-alma9', 'kvm-alma10', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-ubuntu22', 'kvm-ubuntu24', 'kvm-debian12', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'vmware-80', 'vmware-80u1', 'vmware-80u2', 'vmware-80u3', 'vmware-80u3e', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xenserver-84', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82', 'xcpng83']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
when qemuEv is passed, it will deploy KVM hyperviosr hosts with qemu-kvm-ev, else it will default to stock qemu.
When basicZone and/or securityGroups are passed it will create a zone of the last type specified (default is Advanced)
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'shwstppr', 'Damans227', 'vishesh92', 'Pearl1594', 'harikrishna-patnala', 'nvazquez', 'DaanHoogland', 'weizhouapache', 'borisstoyanov', 'vladimirpetrov', 'kiranchavala', 'andrijapanicsb', 'NuxRo', 'rajujith', 'sureshanaparti', 'abh1sar', 'sudo87', 'RosiKyu']