KVM Agent config to reserve dom0 CPUs #7987

mlsorensen · 2023-09-20T15:57:03Z

Description

This PR allows an admin to reserve some hypervisor host CPUs for system use. Another way to think of it is limiting the number of CPUs allocatable to VMs. This can be useful if the admin wants to do other things with the hypervisor's CPU, for example reserve some cores for running hyperconverged storage processes.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Unit test provided, tested locally as well by setting host.reserved.cpu.count to various settings, restarting agent, confirming at UI that host's CPU count has changed.

codecov · 2023-09-20T16:46:21Z

Codecov Report

Merging #7987 (8bbc06c) into main (3c60829) will increase coverage by 0.99%.
The diff coverage is 61.53%.

@@             Coverage Diff              @@
##               main    #7987      +/-   ##
============================================
+ Coverage     28.27%   29.26%   +0.99%     
- Complexity    29328    30489    +1161     
============================================
  Files          5100     5100              
  Lines        358336   358343       +7     
  Branches      52311    52312       +1     
============================================
+ Hits         101305   104859    +3554     
+ Misses       243016   239084    -3932     
- Partials      14015    14400     +385

Flag	Coverage Δ
simulator-marvin-tests	`25.23% <0.00%> (+1.24%)`	⬆️
uitests	`4.87% <ø> (ø)`
unit-tests	`14.50% <66.66%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...va/com/cloud/agent/properties/AgentProperties.java	`0.00% <0.00%> (ø)`
...org/apache/cloudstack/utils/linux/KVMHostInfo.java	`58.58% <88.88%> (+2.20%)`	⬆️
...ervisor/kvm/resource/LibvirtComputingResource.java	`18.68% <0.00%> (-0.01%)`	⬇️

... and 253 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

...ypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java

GabrielBrascher

LGTM

DaanHoogland

looks good (but aggree with @GutoVeronezi 's remark)

rohityadavcloud · 2023-09-25T14:53:17Z

@blueorangutan package

blueorangutan · 2023-09-25T14:54:03Z

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-09-25T15:57:18Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7098

rohityadavcloud · 2023-09-26T09:24:37Z

@blueorangutan test

blueorangutan · 2023-09-26T09:26:03Z

@rohityadavcloud a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-09-27T01:28:57Z

[SF] Trillian test result (tid-7720)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56336 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7987-t7720-kvm-centos7.zip
Smoke tests completed. 112 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_invalid_upgrade_kubernetes_cluster	`Failure`	3616.55	test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster	`Failure`	3608.40	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster	`Failure`	47.69	test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle	`Error`	1.21	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	69.73	test_kubernetes_clusters.py

rohityadavcloud · 2023-09-27T07:07:10Z

It seems the k8s/cks failures not related to this PR, merging based on packaging, smoketests. Looking at intermittent smoketests failures in github actions, closing-reopening to rekick them.

github-actions · 2023-09-27T07:14:10Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions · 2023-09-27T07:34:38Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions · 2023-09-27T09:58:55Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

rohityadavcloud · 2023-09-27T11:32:21Z

@mlsorensen can you help resolve the conflict?

mlsorensen · 2023-09-27T20:34:57Z

Rebased for LibvirtComputingResource style changes.

rohityadavcloud · 2023-09-28T06:40:17Z

@blueorangutan package

blueorangutan · 2023-09-28T06:42:04Z

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-09-28T07:48:40Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7146

DaanHoogland · 2023-09-28T08:11:26Z

@blueorangutan test

blueorangutan · 2023-09-28T08:12:03Z

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

rohityadavcloud · 2023-09-29T06:48:05Z

@blueorangutan test

blueorangutan · 2023-09-29T06:50:05Z

@rohityadavcloud a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-09-29T22:02:10Z

[SF] Trillian test result (tid-7768)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 53230 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7987-t7768-kvm-centos7.zip
Smoke tests completed. 111 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_invalid_upgrade_kubernetes_cluster	`Failure`	3619.93	test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster	`Failure`	3621.11	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster	`Failure`	0.03	test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster	`Failure`	48.79	test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle	`Error`	1.23	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	54.42	test_kubernetes_clusters.py
test_05_vmschedule_test_e2e	`Failure`	291.01	test_vm_schedule.py

DaanHoogland · 2023-10-02T06:00:06Z

@blueorangutan test rocky8 kvm-rocky8

blueorangutan · 2023-10-02T06:02:03Z

@DaanHoogland a [SF] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

blueorangutan · 2023-10-02T22:35:08Z

[SF] Trillian test result (tid-7793)
Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8
Total time taken: 57548 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7987-t7793-kvm-rocky8.zip
Smoke tests completed. 110 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_invalid_upgrade_kubernetes_cluster	`Failure`	3604.51	test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster	`Failure`	3620.52	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	`Failure`	0.06	test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster	`Failure`	0.06	test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	0.06	test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster	`Failure`	50.29	test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle	`Error`	0.28	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	55.08	test_kubernetes_clusters.py
test_05_vmschedule_test_e2e	`Failure`	331.51	test_vm_schedule.py
test_disable_oobm_ha_state_ineligible	`Error`	1516.07	test_hostha_kvm.py

rohityadavcloud · 2023-10-04T06:51:07Z

@blueorangutan test rocky8 kvm-rocky8

blueorangutan · 2023-10-04T06:52:04Z

@rohityadavcloud a [SF] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

blueorangutan · 2023-10-04T20:19:23Z

[SF] Trillian test result (tid-7833)
Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8
Total time taken: 46463 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7987-t7833-kvm-rocky8.zip
Smoke tests completed. 112 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_upgrade_kubernetes_cluster	`Failure`	587.61	test_kubernetes_clusters.py

GutoVeronezi

CLGTM

boring-cyborg bot added component:agent component:kvm labels Sep 20, 2023

GutoVeronezi reviewed Sep 20, 2023

View reviewed changes

...ypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java Outdated Show resolved Hide resolved

GabrielBrascher approved these changes Sep 20, 2023

View reviewed changes

DaanHoogland approved these changes Sep 21, 2023

View reviewed changes

rohityadavcloud added this to the 4.19.0.0 milestone Sep 25, 2023

rohityadavcloud approved these changes Sep 27, 2023

View reviewed changes

rohityadavcloud closed this Sep 27, 2023

rohityadavcloud reopened this Sep 27, 2023

github-actions bot added the status:has-conflicts label Sep 27, 2023

KVM Agent config to reserve dom0 CPUs

8bbc06c

mlsorensen force-pushed the main-host-reserved-dom0-cpu branch from aabee48 to 8bbc06c Compare September 27, 2023 20:33

github-actions bot removed the status:has-conflicts label Sep 27, 2023

shwstppr requested a review from GutoVeronezi October 5, 2023 08:21

GutoVeronezi approved these changes Oct 5, 2023

View reviewed changes

shwstppr merged commit 82b9818 into apache:main Oct 6, 2023
25 of 26 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KVM Agent config to reserve dom0 CPUs #7987

KVM Agent config to reserve dom0 CPUs #7987

mlsorensen commented Sep 20, 2023

codecov bot commented Sep 20, 2023 •

edited

GabrielBrascher left a comment

DaanHoogland left a comment

rohityadavcloud commented Sep 25, 2023

blueorangutan commented Sep 25, 2023

blueorangutan commented Sep 25, 2023

rohityadavcloud commented Sep 26, 2023

blueorangutan commented Sep 26, 2023

blueorangutan commented Sep 27, 2023

rohityadavcloud commented Sep 27, 2023 •

edited

github-actions bot commented Sep 27, 2023

github-actions bot commented Sep 27, 2023

github-actions bot commented Sep 27, 2023

rohityadavcloud commented Sep 27, 2023

mlsorensen commented Sep 27, 2023

rohityadavcloud commented Sep 28, 2023

blueorangutan commented Sep 28, 2023

blueorangutan commented Sep 28, 2023

DaanHoogland commented Sep 28, 2023

blueorangutan commented Sep 28, 2023

rohityadavcloud commented Sep 29, 2023

blueorangutan commented Sep 29, 2023

blueorangutan commented Sep 29, 2023

DaanHoogland commented Oct 2, 2023

blueorangutan commented Oct 2, 2023

blueorangutan commented Oct 2, 2023

rohityadavcloud commented Oct 4, 2023

blueorangutan commented Oct 4, 2023

blueorangutan commented Oct 4, 2023

GutoVeronezi left a comment

KVM Agent config to reserve dom0 CPUs #7987

KVM Agent config to reserve dom0 CPUs #7987

Conversation

mlsorensen commented Sep 20, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

codecov bot commented Sep 20, 2023 • edited

Codecov Report

GabrielBrascher left a comment

Choose a reason for hiding this comment

DaanHoogland left a comment

Choose a reason for hiding this comment

rohityadavcloud commented Sep 25, 2023

blueorangutan commented Sep 25, 2023

blueorangutan commented Sep 25, 2023

rohityadavcloud commented Sep 26, 2023

blueorangutan commented Sep 26, 2023

blueorangutan commented Sep 27, 2023

rohityadavcloud commented Sep 27, 2023 • edited

github-actions bot commented Sep 27, 2023

github-actions bot commented Sep 27, 2023

github-actions bot commented Sep 27, 2023

rohityadavcloud commented Sep 27, 2023

mlsorensen commented Sep 27, 2023

rohityadavcloud commented Sep 28, 2023

blueorangutan commented Sep 28, 2023

blueorangutan commented Sep 28, 2023

DaanHoogland commented Sep 28, 2023

blueorangutan commented Sep 28, 2023

rohityadavcloud commented Sep 29, 2023

blueorangutan commented Sep 29, 2023

blueorangutan commented Sep 29, 2023

DaanHoogland commented Oct 2, 2023

blueorangutan commented Oct 2, 2023

blueorangutan commented Oct 2, 2023

rohityadavcloud commented Oct 4, 2023

blueorangutan commented Oct 4, 2023

blueorangutan commented Oct 4, 2023

GutoVeronezi left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 20, 2023 •

edited

rohityadavcloud commented Sep 27, 2023 •

edited