New Feature: Import VMware VMs into KVM #7881

nvazquez · 2023-08-19T23:15:51Z

Description

This PR adds the capability in CloudStack to convert VMware Instances disk(s) to KVM using virt-v2v and import them as CloudStack instances. It enables CloudStack operators to import VMware instances from vSphere into a KVM cluster managed by CloudStack. vSphere/VMware setup might be managed by CloudStack or be a standalone setup.

CloudStack will let the administrator select a VM from an existing VMware vCenter in the CloudStack environment or external vCenter requesting vCenter IP, Datacenter name and credentials.
The migrated VM will be imported as a KVM instance
The migration is done through virt-v2v: https://access.redhat.com/articles/1351473, https://www.ovirt.org/develop/release-management/features/virt/virt-v2v-integration.html
The migration process timeout can be set by the setting convert.instance.process.timeout
Before attempting the virt-v2v migration, CloudStack will create a clone of the source VM on VMware. The clone VM will be removed after the registration process finishes.
CloudStack will delegate the migration action to a KVM host and the host will attempt to migrate the VM invoking virt-v2v. In case the guest OS is not supported then CloudStack will handle the error operation as a failure
The migration process using virt-v2v may not be a fast process
CloudStack will not perform any check about the guest OS compatibility for the virt-v2v library as indicated on: https://access.redhat.com/articles/1351473.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Create a VMware CloudStack environment
Create single disk and multidisk VMs on CloudStack, start and stop them.
Create another KVM CloudStack environment
import VMs from Vmware to KVM
Deploy VMs

Docs PR: apache/cloudstack-documentation#342

nvazquez · 2023-08-19T23:16:22Z

@blueorangutan package

blueorangutan · 2023-08-19T23:18:02Z

@nvazquez a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2023-08-20T00:06:43Z

Codecov Report

Merging #7881 (5784dad) into main (121531e) will decrease coverage by 9.17%.
The diff coverage is 0.61%.

@@             Coverage Diff              @@
##               main    #7881      +/-   ##
============================================
- Coverage     29.19%   20.02%   -9.17%     
+ Complexity    31015    19769   -11246     
============================================
  Files          5181     5056     -125     
  Lines        365255   344074   -21181     
  Branches      53427    49562    -3865     
============================================
- Hits         106646    68914   -37732     
- Misses       243988   265110   +21122     
+ Partials      14621    10050    -4571

Flag	Coverage Δ
simulator-marvin-tests	`21.40% <0.77%> (-3.76%)`	⬇️
uitests	`4.47% <0.00%> (-0.03%)`	⬇️
unit-tests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...main/java/com/cloud/hypervisor/HypervisorGuru.java	`100.00% <ø> (ø)`
...n/java/org/apache/cloudstack/api/ApiConstants.java	`40.90% <ø> (ø)`
...udstack/api/response/VmwareDatacenterResponse.java	`0.00% <ø> (ø)`
...rc/main/java/com/cloud/storage/StorageManager.java	`80.64% <100.00%> (-16.03%)`	⬇️
...src/main/java/com/cloud/dc/VmwareDatacenterVO.java	`7.89% <ø> (ø)`
...hypervisor/vmware/dao/VmwareDatacenterDaoImpl.java	`54.54% <ø> (ø)`
...ain/java/com/cloud/storage/StorageManagerImpl.java	`21.81% <ø> (-4.07%)`	⬇️
ui/src/components/widgets/TooltipLabel.vue	`100.00% <ø> (ø)`
...va/com/cloud/agent/properties/AgentProperties.java	`0.00% <0.00%> (ø)`
ui/src/views/compute/DeployVM.vue	`0.00% <0.00%> (ø)`
... and 24 more

... and 1583 files with indirect coverage changes

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

blueorangutan · 2023-08-20T00:08:31Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6830

nvazquez · 2023-08-20T01:06:24Z

@blueorangutan test

blueorangutan · 2023-08-20T01:08:02Z

@nvazquez a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-08-20T14:32:13Z

[SF] Trillian test result (tid-7480)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46813 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7881-t7480-kvm-centos7.zip
Smoke tests completed. 110 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestISOUsage>:setup	`Error`	0.00	test_usage.py
test_01_template_usage	`Error`	12.41	test_usage.py
test_02_upgrade_kubernetes_cluster	`Failure`	541.70	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	847.20	test_kubernetes_clusters.py
test_01_create_template	`Error`	12.41	test_templates.py
test_CreateTemplateWithDuplicateName	`Error`	22.73	test_templates.py
test_03_delete_template	`Error`	1.07	test_templates.py

github-actions · 2023-08-22T11:47:18Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

nvazquez · 2023-08-24T02:44:40Z

@blueorangutan package

blueorangutan · 2023-08-24T02:46:03Z

@nvazquez a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-08-24T03:43:51Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6874

nvazquez · 2023-08-26T17:59:01Z

@blueorangutan package

blueorangutan · 2023-12-03T18:34:03Z

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-12-03T20:05:03Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7900

github-actions · 2023-12-04T09:59:20Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

nvazquez · 2023-12-04T11:39:03Z

@blueorangutan package

blueorangutan · 2023-12-04T11:40:05Z

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

ui/public/locales/en.json

andrijapanicsb

LGTM after review and manual testing

borisstoyanov

LGTM, manually tested it

NuxRo

LGTM

nvazquez · 2023-12-06T12:14:01Z

@blueorangutan package

DaanHoogland · 2023-12-06T14:12:25Z

@shwstppr , i think we are done here

shwstppr · 2023-12-06T15:36:46Z

@DaanHoogland yes but the last test run is from 7 days back. Should we run a new round and merge tomorrow if all is good?

shwstppr · 2023-12-06T15:37:07Z

@blueorangutan package

blueorangutan · 2023-12-06T15:38:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-12-06T16:47:25Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7948

shwstppr · 2023-12-06T16:54:13Z

@blueorangutan test

blueorangutan · 2023-12-06T16:56:03Z

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-12-07T06:15:04Z

[SF] Trillian test result (tid-8489)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46383 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7881-t8489-kvm-centos7.zip
Smoke tests completed. 120 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_migrate_vm	`Error`	0.06	test_vm_life_cycle.py

This PR adds the capability in CloudStack to convert VMware Instances disk(s) to KVM using virt-v2v and import them as CloudStack instances. It enables CloudStack operators to import VMware instances from vSphere into a KVM cluster managed by CloudStack. vSphere/VMware setup might be managed by CloudStack or be a standalone setup. CloudStack will let the administrator select a VM from an existing VMware vCenter in the CloudStack environment or external vCenter requesting vCenter IP, Datacenter name and credentials. The migrated VM will be imported as a KVM instance The migration is done through virt-v2v: https://access.redhat.com/articles/1351473, https://www.ovirt.org/develop/release-management/features/virt/virt-v2v-integration.html The migration process timeout can be set by the setting convert.instance.process.timeout Before attempting the virt-v2v migration, CloudStack will create a clone of the source VM on VMware. The clone VM will be removed after the registration process finishes. CloudStack will delegate the migration action to a KVM host and the host will attempt to migrate the VM invoking virt-v2v. In case the guest OS is not supported then CloudStack will handle the error operation as a failure The migration process using virt-v2v may not be a fast process CloudStack will not perform any check about the guest OS compatibility for the virt-v2v library as indicated on: https://access.redhat.com/articles/1351473.

Documentation for Import VMware VMs into KVM PR: apache/cloudstack#7881

nvazquez added component:kvm component:vmware component:templates component:migration labels Aug 19, 2023

boring-cyborg bot added component:api component:orchestration component:packaging component:storage distro:centos distro:ubuntu labels Aug 19, 2023

boring-cyborg bot added the component:backup label Aug 22, 2023

github-actions bot added the status:has-conflicts label Aug 22, 2023

nvazquez changed the title ~~[WIP] New Feature: Register KVM templates from migrated VMware VM~~ [WIP] New Feature: Migrate VMware VMs to KVM Aug 24, 2023

github-actions bot removed the status:has-conflicts label Aug 24, 2023

New Feature: Migrate VMware VMs to KVM

f327792

nvazquez force-pushed the migrate-vmw-to-kvm branch from 9b1ac3a to f327792 Compare August 24, 2023 02:48

nvazquez added 2 commits August 26, 2023 14:23

Merge branch 'main' into migrate-vmw-to-kvm

2dc451a

Fix delete template endpoint

bd8859a

github-actions bot added the status:has-conflicts label Dec 4, 2023

Merge branch 'main' into migrate-vmw-to-kvm

8460850

github-actions bot removed the status:has-conflicts label Dec 4, 2023

andrijapanicsb reviewed Dec 6, 2023

View reviewed changes

ui/public/locales/en.json Outdated Show resolved Hide resolved

andrijapanicsb marked this pull request as ready for review December 6, 2023 10:51

andrijapanicsb self-requested a review December 6, 2023 10:52

andrijapanicsb approved these changes Dec 6, 2023

View reviewed changes

andrijapanicsb requested review from borisstoyanov and NuxRo December 6, 2023 10:55

borisstoyanov reviewed Dec 6, 2023

View reviewed changes

NuxRo approved these changes Dec 6, 2023

View reviewed changes

Improve import export header description

746ed3c

shwstppr merged commit 371ad9f into apache:main Dec 7, 2023
24 of 26 checks passed

apache deleted a comment from shwstppr Dec 7, 2023

shwstppr pushed a commit to apache/cloudstack-documentation that referenced this pull request Jan 17, 2024

Import VMware VMs into KVM - Documentation (#342)

cc5ee97

Documentation for Import VMware VMs into KVM PR: apache/cloudstack#7881

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Feature: Import VMware VMs into KVM #7881

New Feature: Import VMware VMs into KVM #7881

nvazquez commented Aug 19, 2023 •

edited by andrijapanicsb

nvazquez commented Aug 19, 2023

blueorangutan commented Aug 19, 2023

codecov bot commented Aug 20, 2023 •

edited

blueorangutan commented Aug 20, 2023

nvazquez commented Aug 20, 2023

blueorangutan commented Aug 20, 2023

blueorangutan commented Aug 20, 2023

github-actions bot commented Aug 22, 2023

nvazquez commented Aug 24, 2023

blueorangutan commented Aug 24, 2023

blueorangutan commented Aug 24, 2023

nvazquez commented Aug 26, 2023

blueorangutan commented Dec 3, 2023

blueorangutan commented Dec 3, 2023

github-actions bot commented Dec 4, 2023

nvazquez commented Dec 4, 2023

blueorangutan commented Dec 4, 2023

andrijapanicsb left a comment

borisstoyanov left a comment

NuxRo left a comment

nvazquez commented Dec 6, 2023

DaanHoogland commented Dec 6, 2023

shwstppr commented Dec 6, 2023

shwstppr commented Dec 6, 2023

blueorangutan commented Dec 6, 2023

blueorangutan commented Dec 6, 2023

shwstppr commented Dec 6, 2023

blueorangutan commented Dec 6, 2023

blueorangutan commented Dec 7, 2023

New Feature: Import VMware VMs into KVM #7881

New Feature: Import VMware VMs into KVM #7881

Conversation

nvazquez commented Aug 19, 2023 • edited by andrijapanicsb

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

nvazquez commented Aug 19, 2023

blueorangutan commented Aug 19, 2023

codecov bot commented Aug 20, 2023 • edited

Codecov Report

blueorangutan commented Aug 20, 2023

nvazquez commented Aug 20, 2023

blueorangutan commented Aug 20, 2023

blueorangutan commented Aug 20, 2023

github-actions bot commented Aug 22, 2023

nvazquez commented Aug 24, 2023

blueorangutan commented Aug 24, 2023

blueorangutan commented Aug 24, 2023

nvazquez commented Aug 26, 2023

blueorangutan commented Dec 3, 2023

blueorangutan commented Dec 3, 2023

github-actions bot commented Dec 4, 2023

nvazquez commented Dec 4, 2023

blueorangutan commented Dec 4, 2023

andrijapanicsb left a comment

Choose a reason for hiding this comment

borisstoyanov left a comment

Choose a reason for hiding this comment

NuxRo left a comment

Choose a reason for hiding this comment

nvazquez commented Dec 6, 2023

DaanHoogland commented Dec 6, 2023

shwstppr commented Dec 6, 2023

shwstppr commented Dec 6, 2023

blueorangutan commented Dec 6, 2023

blueorangutan commented Dec 6, 2023

shwstppr commented Dec 6, 2023

blueorangutan commented Dec 6, 2023

blueorangutan commented Dec 7, 2023

nvazquez commented Aug 19, 2023 •

edited by andrijapanicsb

codecov bot commented Aug 20, 2023 •

edited