Add support for vSphere 8.0 #7046

rohityadavcloud · 2023-01-04T06:30:33Z

Adds basic support to bump VMware vim sdk to 8.0, enable in hypervisor capabilties and copy guest mappings from the previous version.

May be a quick one to accept this, to have early access/support for 8.0 in ACS. Later any integration that requires changes in ACS for support in Trillian/BO can be done as separate PR.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud · 2023-01-04T06:59:25Z

New SDK appears not backward-compatible :(

12:25:08 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project cloud-vmware-base: Compilation failure
12:25:08 [ERROR] /jenkins/workspace/acs-deb-pkg-builder/vmware-base/src/main/java/com/cloud/hypervisor/vmware/mo/VirtualStorageObjectManagerMO.java:[61,36] error: method retrieveVStorageObject in interface VimPortType cannot be applied to given types;
12:25:08 [ERROR]   required: ManagedObjectReference,ID,ManagedObjectReference,List<String>
12:25:08 [ERROR]   found: ManagedObjectReference,ID,ManagedObjectReference
12:25:08 [ERROR]   reason: actual and formal argument lists differ in length

Based on vSphere8 SDK docs and samples, doesn't seem we need to pass anything to it.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud · 2023-01-04T07:13:00Z

@blueorangutan package

blueorangutan · 2023-01-04T07:14:05Z

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-01-04T08:02:10Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5165

weizhouapache · 2023-01-04T08:51:29Z

@rohityadavcloud
good start !
will you work on the ESXi and VCSA templates for trillian tests as well ?

codecov · 2023-01-04T10:20:17Z

Codecov Report

Merging #7046 (e108439) into main (af59e76) will increase coverage by 0.31%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##               main    #7046      +/-   ##
============================================
+ Coverage     11.54%   11.85%   +0.31%     
- Complexity     7509     7898     +389     
============================================
  Files          2494     2514      +20     
  Lines        246678   258550   +11872     
  Branches      38533    43354    +4821     
============================================
+ Hits          28476    30651    +2175     
- Misses       214521   223889    +9368     
- Partials       3681     4010     +329

Impacted Files	Coverage Δ
...visor/vmware/mo/VirtualStorageObjectManagerMO.java	`0.00% <0.00%> (ø)`
...er/src/main/java/com/cloud/hypervisor/KVMGuru.java	`62.15% <0.00%> (-11.08%)`	⬇️
...ngine/schema/src/main/java/com/cloud/vm/NicVO.java	`50.54% <0.00%> (-1.42%)`	⬇️
...ema/src/main/java/com/cloud/vm/dao/NicDaoImpl.java	`23.38% <0.00%> (-0.97%)`	⬇️
...n/java/com/cloud/network/dao/IPAddressDaoImpl.java	`30.83% <0.00%> (-0.50%)`	⬇️
.../java/com/cloud/network/guru/GuestNetworkGuru.java	`32.13% <0.00%> (-0.47%)`	⬇️
...n/java/com/cloud/vm/VirtualMachineManagerImpl.java	`5.46% <0.00%> (-0.44%)`	⬇️
...ork/router/VirtualNetworkApplianceManagerImpl.java	`2.43% <0.00%> (-0.31%)`	⬇️
...com/cloud/api/query/dao/DataCenterJoinDaoImpl.java	`1.75% <0.00%> (-0.21%)`	⬇️
.../java/com/cloud/hypervisor/HypervisorGuruBase.java	`9.83% <0.00%> (-0.17%)`	⬇️
... and 54 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

rohityadavcloud · 2023-01-04T11:24:24Z

I'll discuss soon @weizhouapache @DaanHoogland - but it may be worth adding support for this in 4.18.0 or 4.18.1 (draft for now), and adding support in Trillian eventually.

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2023-01-04T11:25:03Z

@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

rohityadavcloud · 2023-01-04T11:25:11Z

@blueorangutan test centos7 vmware-70u3

blueorangutan · 2023-01-04T11:26:03Z

@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + vmware-70u3) has been kicked to run smoke tests

blueorangutan · 2023-01-05T00:40:35Z

Trillian test result (tid-5745)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46174 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5745-vmware-70u3.zip
Smoke tests completed. 106 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2023-01-05T03:35:06Z

Trillian test result (tid-5744)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 57065 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5744-vmware-67u3.zip
Smoke tests completed. 104 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	654.18	test_kubernetes_clusters.py
test_01_deploy_vm_on_specific_host	`Error`	3603.46	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	`Error`	2.39	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	`Error`	2.34	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	`Error`	1.40	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	`Error`	3.35	test_vm_deployment_planner.py

rohityadavcloud · 2023-01-09T11:37:47Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2023-01-09T11:38:03Z

@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2023-01-10T01:11:29Z

Trillian test result (tid-5765)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47668 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5765-vmware-67u3.zip
Smoke tests completed. 105 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_07_arping_in_vr	`Failure`	5.22	test_diagnostics.py

rohityadavcloud · 2023-01-12T09:53:30Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2023-01-12T22:59:12Z

Trillian test result (tid-5810)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43932 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5810-vmware-65u2.zip
Smoke tests completed. 106 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2023-01-12T23:32:18Z

Trillian test result (tid-5809)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47957 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5809-vmware-67u3.zip
Smoke tests completed. 105 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	620.96	test_kubernetes_clusters.py

blueorangutan · 2023-01-13T02:45:14Z

Trillian test result (tid-5811)
Environment: vmware-60u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 57433 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5811-vmware-60u2.zip
Smoke tests completed. 104 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	666.32	test_kubernetes_clusters.py
test_01_deploy_vm_on_specific_host	`Error`	18.73	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	`Error`	3602.38	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	`Error`	4.43	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	`Error`	3.81	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	`Error`	4.41	test_vm_deployment_planner.py

DaanHoogland · 2023-01-13T05:23:05Z

just to be sure
@blueorangutan test centos7 vmware-60u2

blueorangutan · 2023-01-13T05:24:03Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + vmware-60u2) has been kicked to run smoke tests

blueorangutan · 2023-01-13T19:37:13Z

Trillian test result (tid-5828)
Environment: vmware-60u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49993 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5828-vmware-60u2.zip
Smoke tests completed. 105 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	665.64	test_kubernetes_clusters.py

DaanHoogland · 2023-01-16T09:16:42Z

@rohityadavcloud , do we need to add guest OS mappings?

engine/schema/src/main/resources/META-INF/db/schema-41720to41800.sql

weizhouapache · 2023-01-17T09:28:24Z

@blueorangutan test ubuntu20 vmware-70u3

blueorangutan · 2023-01-17T09:29:03Z

@weizhouapache a Trillian-Jenkins test job (ubuntu20 mgmt + vmware-70u3) has been kicked to run smoke tests

rohityadavcloud · 2023-01-18T05:16:50Z

@blueorangutan test ubuntu20 vmware-70u3

blueorangutan · 2023-01-18T05:17:03Z

@rohityadavcloud a Trillian-Jenkins test job (ubuntu20 mgmt + vmware-70u3) has been kicked to run smoke tests

weizhouapache · 2023-01-18T06:24:35Z

@rohityadavcloud
The trillian failures are not related, no worries

DaanHoogland · 2023-01-18T10:57:46Z

@weizhouapache @rohityadavcloud are we in agreement on what to do with this, yet? fine by me either way (as RM).

rohityadavcloud · 2023-01-18T14:44:06Z

Did basic deployment with ESXi 8.0.0, vCenter 8.0a; zone deployment and system VM lifecycle was OK. @weizhouapache @DaanHoogland

weizhouapache

great, thanks @rohityadavcloud

weizhouapache · 2023-01-18T14:51:42Z

@weizhouapache @rohityadavcloud are we in agreement on what to do with this, yet? fine by me either way (as RM).

@DaanHoogland
let's merge it

weizhouapache · 2023-01-18T15:38:56Z

@blueorangutan package

blueorangutan · 2023-01-18T15:40:03Z

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-01-18T16:40:27Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5315

blueorangutan · 2023-01-18T19:56:51Z

Trillian test result (tid-5882)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server u20
Total time taken: 51111 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5882-vmware-70u3.zip
Smoke tests completed. 95 look OK, 11 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_verify_ipv6_network	`Failure`	113.46	test_network_ipv6.py
test_isolate_network_password_server	`Failure`	11.67	test_password_server.py
test_router_dhcphosts	`Failure`	16.18	test_router_dhcphosts.py
test_router_dhcp_opts	`Failure`	11.96	test_router_dhcphosts.py
test_03_ssvm_internals	`Failure`	0.72	test_ssvm.py
test_04_cpvm_internals	`Failure`	0.67	test_ssvm.py
test_05_stop_ssvm	`Failure`	82.79	test_ssvm.py
test_06_stop_cpvm	`Failure`	82.51	test_ssvm.py
test_07_reboot_ssvm	`Failure`	98.15	test_ssvm.py
test_08_reboot_cpvm	`Failure`	23.39	test_ssvm.py
test_09_reboot_ssvm_forced	`Failure`	165.21	test_ssvm.py
test_10_reboot_cpvm_forced	`Failure`	64.74	test_ssvm.py
test_11_destroy_ssvm	`Failure`	70.83	test_ssvm.py
test_12_destroy_cpvm	`Failure`	63.87	test_ssvm.py
test_network_rules_acquired_public_ip_1_static_nat_rule	`Failure`	5.31	test_network.py
test_network_rules_acquired_public_ip_2_nat_rule	`Failure`	6.10	test_network.py
test_network_rules_acquired_public_ip_3_Load_Balancer_Rule	`Failure`	8.18	test_network.py
test_01_deployVMInSharedNetwork	`Failure`	98.92	test_network.py
test_02_verifyRouterIpAfterNetworkRestart	`Failure`	52.27	test_network.py
test_03_destroySharedNetwork	`Failure`	1.10	test_network.py
ContextSuite context=TestSharedNetwork>:teardown	`Error`	2.23	test_network.py
test_01_router_internal_basic	`Failure`	0.81	test_routers.py
test_02_router_internal_adv	`Failure`	0.68	test_routers.py
test_04_restart_network_wo_cleanup	`Failure`	3.00	test_routers.py
test_03_RVR_Network_check_router_state	`Failure`	230.86	test_routers_network_ops.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	666.43	test_kubernetes_clusters.py
test_02_routervm_iptables_policies	`Failure`	101.77	test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies	`Failure`	270.05	test_routers_iptables_default_policy.py
test_01_verify_ipv6_vpc	`Failure`	179.26	test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	480.59	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	`Failure`	479.83	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	368.91	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	`Failure`	305.86	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	439.58	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	439.60	test_vpc_redundant.py

DaanHoogland · 2023-01-19T08:46:04Z

hm, doing an extra check, I don´t think these errors are related to this PR but
@blueorangutan test ubuntu20 vmware-70u3

blueorangutan · 2023-01-19T08:47:03Z

@DaanHoogland a Trillian-Jenkins test job (ubuntu20 mgmt + vmware-70u3) has been kicked to run smoke tests

rohityadavcloud · 2023-01-19T09:35:26Z

Ubuntu mgmt server smoketests doesn’t work with VMware, Wei logged a bug or PR on trillian I think

weizhouapache · 2023-01-19T09:37:07Z

Ubuntu mgmt server smoketests doesn’t work with VMware, Wei logged a bug or PR on trillian I think

yes @rohityadavcloud
It is actually not a bug with Trillian, but with utils in tools/marvin. I will create a PR to fix it

blueorangutan · 2023-01-19T23:39:21Z

Trillian test result (tid-5904)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server u20
Total time taken: 51800 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7046-t5904-vmware-70u3.zip
Smoke tests completed. 96 look OK, 11 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_verify_ipv6_network	`Failure`	113.53	test_network_ipv6.py
test_03_ssvm_internals	`Failure`	0.66	test_ssvm.py
test_04_cpvm_internals	`Failure`	0.65	test_ssvm.py
test_05_stop_ssvm	`Failure`	81.55	test_ssvm.py
test_06_stop_cpvm	`Failure`	82.53	test_ssvm.py
test_07_reboot_ssvm	`Failure`	99.18	test_ssvm.py
test_08_reboot_cpvm	`Failure`	23.43	test_ssvm.py
test_09_reboot_ssvm_forced	`Failure`	164.02	test_ssvm.py
test_10_reboot_cpvm_forced	`Failure`	62.74	test_ssvm.py
test_11_destroy_ssvm	`Failure`	76.99	test_ssvm.py
test_12_destroy_cpvm	`Failure`	63.70	test_ssvm.py
test_router_dhcphosts	`Failure`	16.40	test_router_dhcphosts.py
test_router_dhcp_opts	`Failure`	11.82	test_router_dhcphosts.py
test_isolate_network_password_server	`Failure`	11.64	test_password_server.py
test_01_router_internal_basic	`Failure`	1.08	test_routers.py
test_02_router_internal_adv	`Failure`	0.75	test_routers.py
test_04_restart_network_wo_cleanup	`Failure`	2.90	test_routers.py
test_network_rules_acquired_public_ip_1_static_nat_rule	`Failure`	5.73	test_network.py
test_network_rules_acquired_public_ip_2_nat_rule	`Failure`	6.17	test_network.py
test_network_rules_acquired_public_ip_3_Load_Balancer_Rule	`Failure`	8.36	test_network.py
test_01_deployVMInSharedNetwork	`Failure`	97.14	test_network.py
test_02_verifyRouterIpAfterNetworkRestart	`Failure`	53.52	test_network.py
test_03_destroySharedNetwork	`Failure`	0.08	test_network.py
ContextSuite context=TestSharedNetwork>:teardown	`Error`	0.22	test_network.py
test_03_RVR_Network_check_router_state	`Failure`	181.92	test_routers_network_ops.py
test_02_upgrade_kubernetes_cluster	`Failure`	572.28	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	619.64	test_kubernetes_clusters.py
test_02_routervm_iptables_policies	`Failure`	97.30	test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies	`Failure`	262.29	test_routers_iptables_default_policy.py
test_01_verify_ipv6_vpc	`Failure`	179.79	test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	482.04	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	`Failure`	471.11	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	355.37	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	`Failure`	306.05	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	434.57	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	434.59	test_vpc_redundant.py

Add support for vSphere 8.0

bb9605a

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud added this to the 4.18.0.0 milestone Jan 4, 2023

boring-cyborg bot added the component:build label Jan 4, 2023

rohityadavcloud added the component:vmware label Jan 4, 2023

apache deleted a comment from blueorangutan Jan 4, 2023

fix build issue with new sdk

e108439

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

DaanHoogland added the type:experimental-feature label Jan 4, 2023

DaanHoogland requested a review from harikrishna-patnala January 4, 2023 10:41

rohityadavcloud marked this pull request as ready for review January 12, 2023 09:52

rohityadavcloud requested review from DaanHoogland and weizhouapache January 12, 2023 09:52

rohityadavcloud closed this Jan 12, 2023

rohityadavcloud reopened this Jan 12, 2023

weizhouapache reviewed Jan 16, 2023

View reviewed changes

engine/schema/src/main/resources/META-INF/db/schema-41720to41800.sql Show resolved Hide resolved

apache deleted a comment from blueorangutan Jan 18, 2023

rohityadavcloud requested a review from weizhouapache January 18, 2023 14:48

weizhouapache approved these changes Jan 18, 2023

View reviewed changes

DaanHoogland merged commit e8c32d6 into apache:main Jan 18, 2023

stephankruggg pushed a commit to scclouds/cloudstack that referenced this pull request Jan 25, 2023

Add support for vSphere 8.0 (apache#7046)

479b335

Add support for vSphere 8.0 #7046

Add support for vSphere 8.0 #7046

Conversation

rohityadavcloud commented Jan 4, 2023 • edited

rohityadavcloud commented Jan 4, 2023 • edited

rohityadavcloud commented Jan 4, 2023

blueorangutan commented Jan 4, 2023

blueorangutan commented Jan 4, 2023

weizhouapache commented Jan 4, 2023

codecov bot commented Jan 4, 2023 • edited

Codecov Report

rohityadavcloud commented Jan 4, 2023

blueorangutan commented Jan 4, 2023

rohityadavcloud commented Jan 4, 2023

blueorangutan commented Jan 4, 2023

blueorangutan commented Jan 5, 2023

blueorangutan commented Jan 5, 2023

rohityadavcloud commented Jan 9, 2023

blueorangutan commented Jan 9, 2023

blueorangutan commented Jan 10, 2023

rohityadavcloud commented Jan 12, 2023

blueorangutan commented Jan 12, 2023

blueorangutan commented Jan 12, 2023

blueorangutan commented Jan 13, 2023

DaanHoogland commented Jan 13, 2023

blueorangutan commented Jan 13, 2023

blueorangutan commented Jan 13, 2023

DaanHoogland commented Jan 16, 2023

weizhouapache commented Jan 17, 2023

blueorangutan commented Jan 17, 2023

rohityadavcloud commented Jan 18, 2023

blueorangutan commented Jan 18, 2023

weizhouapache commented Jan 18, 2023

DaanHoogland commented Jan 18, 2023

rohityadavcloud commented Jan 18, 2023

weizhouapache left a comment

Choose a reason for hiding this comment

weizhouapache commented Jan 18, 2023

weizhouapache commented Jan 18, 2023

blueorangutan commented Jan 18, 2023

blueorangutan commented Jan 18, 2023

blueorangutan commented Jan 18, 2023

DaanHoogland commented Jan 19, 2023

blueorangutan commented Jan 19, 2023

rohityadavcloud commented Jan 19, 2023 • edited

weizhouapache commented Jan 19, 2023

blueorangutan commented Jan 19, 2023

rohityadavcloud commented Jan 4, 2023 •

edited

rohityadavcloud commented Jan 4, 2023 •

edited

codecov bot commented Jan 4, 2023 •

edited

rohityadavcloud commented Jan 19, 2023 •

edited