smoke test: fixes test_vm_deployment_planner #7718

DaanHoogland · 2023-07-04T14:01:07Z

Description

This PR...

This doesn't fix #7717 , but removes the abundant requirement from test_vm_deployment_planner, which looks for a specific template using the wrong hypervisor. The described issue in #7717 remains that the testclient doesn't have the right type of hypervisor configured.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

weizhouapache · 2023-07-06T09:12:07Z

@blueorangutan package

blueorangutan · 2023-07-06T09:14:03Z

@weizhouapache a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-07-06T10:07:52Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6427

DaanHoogland · 2023-07-10T07:27:03Z

@blueorangutan test rocky8 vmware-67u3

DaanHoogland · 2023-07-10T08:04:57Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2023-07-10T08:06:03Z

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

DaanHoogland · 2023-07-11T12:50:31Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2023-07-11T13:00:04Z

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2023-07-12T01:30:34Z

[SF] Trillian test result (tid-7053)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43714 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7718-t7053-vmware-67u3.zip
Smoke tests completed. 112 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_05_vmschedule_test_e2e	`Failure`	170.64	test_vm_schedule.py

DaanHoogland · 2023-07-12T07:56:32Z

@blueorangutan package

blueorangutan · 2023-07-12T07:58:03Z

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-07-12T08:52:13Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6468

vishesh92 · 2023-07-12T11:36:25Z

@blueorangutan test rocky8 vmware-67u3

DaanHoogland · 2023-07-17T07:38:39Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2023-07-17T07:40:04Z

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2023-07-18T20:36:09Z

[SF] Trillian test result (tid-7100)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44589 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7718-t7100-vmware-67u3.zip
Smoke tests completed. 113 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2023-07-19T08:26:59Z

@shwstppr @weizhouapache can you review?

DaanHoogland · 2023-07-27T08:33:11Z

Same code hypervisor=self.hypervisor is used across different tests so I'm not completely sure about this change

agree with @shwstppr . code looks ok. but I am also wondering if we need this change and what the benefit is.

The real issue is #7717; the hypervisor is not set correctly in the test client, but for these tests it is irrelevant so no reason to set it. Just getting the template is effectively the same action. a test env with a mixed zone may not be able to execute it but only if the hypervisor type is explicitely set to one for which a template is not of the default hypervisortype. The test assumes intelligence that it doesn't have itself.

@shwstppr @weizhouapache , is this explanation enough?

DaanHoogland · 2023-07-31T06:36:13Z

@blueorangutan package

blueorangutan · 2023-07-31T06:38:03Z

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

weizhouapache · 2023-07-31T07:26:49Z

Same code hypervisor=self.hypervisor is used across different tests so I'm not completely sure about this change

agree with @shwstppr . code looks ok. but I am also wondering if we need this change and what the benefit is.

The real issue is #7717; the hypervisor is not set correctly in the test client, but for these tests it is irrelevant so no reason to set it. Just getting the template is effectively the same action. a test env with a mixed zone may not be able to execute it but only if the hypervisor type is explicitely set to one for which a template is not of the default hypervisortype. The test assumes intelligence that it doesn't have itself.

@shwstppr @weizhouapache , is this explanation enough?

@DaanHoogland
I still do not understand what the PR aims to fix.
In mixed zone, if the hypervisor is incorrect, we can update test_data.py to use correct hypervisor.

with this PR, the test will get the template with random (?) hypervisor, right ?

blueorangutan · 2023-07-31T07:43:57Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6606

DaanHoogland · 2023-07-31T09:12:49Z

with this PR, the test will get the template with random (?) hypervisor, right ?

you can call it random, but in an env with only one hypervisor it will get the template registered instead of trying to force to get the kvm template. this is why the tests would fail on vmware.

weizhouapache · 2023-07-31T09:28:43Z

@blueorangutan test rocky8 vmware-67u3

weizhouapache · 2023-07-31T09:29:14Z

with this PR, the test will get the template with random (?) hypervisor, right ?

you can call it random, but in an env with only one hypervisor it will get the template registered instead of trying to force to get the kvm template. this is why the tests would fail on vmware.

ok, let's run some tests on vmware

weizhouapache · 2023-07-31T09:36:17Z

@blueorangutan test rocky8 vmware-67u3

blueorangutan · 2023-07-31T09:38:03Z

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + vmware-67u3) has been kicked to run smoke tests

weizhouapache · 2023-07-31T09:39:01Z

@blueorangutan test ubuntu22 vmware-70u3

weizhouapache · 2023-07-31T09:46:43Z

@blueorangutan test ubuntu22 vmware-70u3

blueorangutan · 2023-07-31T09:48:03Z

@weizhouapache a [SF] Trillian-Jenkins test job (ubuntu22 mgmt + vmware-70u3) has been kicked to run smoke tests

blueorangutan · 2023-08-01T00:16:32Z

[SF] Trillian test result (tid-7217)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server u22
Total time taken: 50312 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7718-t7217-vmware-70u3.zip
Smoke tests completed. 113 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2023-08-01T07:19:01Z

[SF] Trillian test result (tid-7216)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 76560 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7718-t7216-vmware-67u3.zip
Smoke tests completed. 109 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	`Error`	544.90	test_internal_lb.py
test_list_system_vms_metrics_history	`Failure`	0.24	test_metrics_api.py
test_list_vms_metrics_admin	`Error`	3604.82	test_metrics_api.py
test_list_vms_metrics_history	`Error`	4.70	test_metrics_api.py
test_list_volumes_metrics_history	`Error`	3606.87	test_metrics_api.py
test_01_deploy_vm_on_specific_host	`Error`	17.74	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	`Error`	3604.30	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	`Error`	1.29	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	`Error`	4.37	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	`Error`	1.28	test_vm_deployment_planner.py
test_09_expunge_vm	`Failure`	424.68	test_vm_life_cycle.py

weizhouapache · 2023-08-02T00:48:47Z

@blueorangutan test rocky8 vmware-67u3

blueorangutan · 2023-08-02T00:50:03Z

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2023-08-02T20:56:01Z

[SF] Trillian test result (tid-7234)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 70921 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7718-t7234-vmware-67u3.zip
Smoke tests completed. 110 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_list_system_vms_metrics_history	`Failure`	0.23	test_metrics_api.py
test_list_vms_metrics_admin	`Error`	3604.32	test_metrics_api.py
test_list_vms_metrics_history	`Error`	5.72	test_metrics_api.py
test_list_volumes_metrics_history	`Error`	3606.52	test_metrics_api.py
test_01_deploy_vm_on_specific_host	`Error`	16.67	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	`Error`	3602.66	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	`Error`	2.38	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	`Error`	1.35	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	`Error`	4.40	test_vm_deployment_planner.py
test_09_expunge_vm	`Failure`	424.60	test_vm_life_cycle.py

weizhouapache · 2023-08-03T00:38:06Z

@blueorangutan test ubuntu22 vmware-70u3

blueorangutan · 2023-08-03T00:40:03Z

@weizhouapache a [SF] Trillian-Jenkins test job (ubuntu22 mgmt + vmware-70u3) has been kicked to run smoke tests

rohityadavcloud

LGTM haven't tested it

blueorangutan · 2023-08-03T13:36:44Z

[SF] Trillian test result (tid-7242)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server u22
Total time taken: 44864 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7718-t7242-vmware-70u3.zip
Smoke tests completed. 113 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

boring-cyborg bot added component:integration-test Python Warning... Python code Ahead! labels Jul 4, 2023

DaanHoogland added this to the 4.18.1.0 milestone Jul 5, 2023

DaanHoogland added the Severity:Critical Critical bug label Jul 5, 2023

This was referenced Jul 6, 2023

[HEALTH] 4.18 LTS Health Check, please don't merge this! #7345

Closed

Add L2 networks to Zones with SG #7719

Merged

apache deleted a comment from blueorangutan Jul 10, 2023

apache deleted a comment from blueorangutan Jul 17, 2023

apache deleted a comment from blueorangutan Jul 18, 2023

DaanHoogland mentioned this pull request Jul 27, 2023

intermittent smoke test failure for kubernetes clusters #7778

Closed

test_vm_deployment_planner: fix using wrong hypervisor

9c4495e

DaanHoogland force-pushed the smoke.test.fixes branch from 34c3ad3 to 9c4495e Compare July 31, 2023 06:35

DaanHoogland marked this pull request as ready for review July 31, 2023 06:38

weizhouapache approved these changes Jul 31, 2023

View reviewed changes

weizhouapache changed the title ~~smoke test fixes~~ smoke test: fixes test_vm_deployment_planner Jul 31, 2023

rohityadavcloud approved these changes Aug 3, 2023

View reviewed changes

rohityadavcloud merged commit 0318e9f into apache:main Aug 7, 2023

rohityadavcloud deleted the smoke.test.fixes branch August 7, 2023 04:50

smoke test: fixes test_vm_deployment_planner #7718

smoke test: fixes test_vm_deployment_planner #7718

Uh oh!

Conversation

DaanHoogland commented Jul 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

weizhouapache commented Jul 6, 2023

Uh oh!

blueorangutan commented Jul 6, 2023

Uh oh!

blueorangutan commented Jul 6, 2023

Uh oh!

DaanHoogland commented Jul 10, 2023

Uh oh!

DaanHoogland commented Jul 10, 2023

Uh oh!

blueorangutan commented Jul 10, 2023

Uh oh!

DaanHoogland commented Jul 11, 2023

Uh oh!

blueorangutan commented Jul 11, 2023

Uh oh!

blueorangutan commented Jul 12, 2023

Uh oh!

DaanHoogland commented Jul 12, 2023

Uh oh!

blueorangutan commented Jul 12, 2023

Uh oh!

blueorangutan commented Jul 12, 2023

Uh oh!

vishesh92 commented Jul 12, 2023

Uh oh!

DaanHoogland commented Jul 17, 2023

Uh oh!

blueorangutan commented Jul 17, 2023

Uh oh!

blueorangutan commented Jul 18, 2023

Uh oh!

DaanHoogland commented Jul 19, 2023

Uh oh!

DaanHoogland commented Jul 27, 2023

Uh oh!

DaanHoogland commented Jul 31, 2023

Uh oh!

blueorangutan commented Jul 31, 2023

Uh oh!

weizhouapache commented Jul 31, 2023

Uh oh!

blueorangutan commented Jul 31, 2023

Uh oh!

DaanHoogland commented Jul 31, 2023

Uh oh!

weizhouapache commented Jul 31, 2023

Uh oh!

weizhouapache commented Jul 31, 2023

Uh oh!

weizhouapache commented Jul 31, 2023

Uh oh!

blueorangutan commented Jul 31, 2023

Uh oh!

weizhouapache commented Jul 31, 2023

Uh oh!

weizhouapache commented Jul 31, 2023

Uh oh!

blueorangutan commented Jul 31, 2023

Uh oh!

blueorangutan commented Aug 1, 2023

Uh oh!

blueorangutan commented Aug 1, 2023

Uh oh!

weizhouapache commented Aug 2, 2023

Uh oh!

DaanHoogland commented Jul 4, 2023 •

edited

Loading