KVM: add vm setting for nic multiqueue number and packed virtqueues #7333

weizhouapache · 2023-03-13T10:13:07Z

Description

This PR adds two vm setting for user vms on KVM

nic multiqueue number
packed virtqueues enabled . optional are true and false (false by default). It requires qemu>=4.2.0 and libvirt >=6.3.0

Tested ok on ubuntu 22 and rocky 8.4

(See test results in Screenshots part)

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Screenshots (if appropriate):

Deploy a vm
There are two new options in "Advanced mode" part.
Start the vm. check qemu process on kvm host

it shows

-netdev tap,fds=57:59:60:61:62:63,id=hostnet0,vhost=on,vhostfds=64:65:66:67:68:69 
-device virtio-net-pci,packed=on,mq=on,vectors=14,netdev=hostnet0,id=net0,mac=02:00:3f:4f:00:0a,bus=pci.0,addr=0x3

(vectors = 2 * queue number + 2)

virsh dumpxml shows <driver queues='6' packed='on'/>
log into the vm, and run some commands

enable multiqueue inside user vm

weizhouapache · 2023-03-13T11:22:41Z

This works on centos7 as well.

but centos7 with tap device does not work. got an exception

org.libvirt.LibvirtException: Unable to create tap device tap0200492f0004: Invalid argument

codecov · 2023-03-14T17:38:46Z

Codecov Report

Merging #7333 (b2a1b9c) into main (f64f7b9) will increase coverage by 0.00%.
The diff coverage is 30.00%.

@@            Coverage Diff            @@
##               main    #7333   +/-   ##
=========================================
  Coverage     12.68%   12.69%           
- Complexity     8654     8656    +2     
=========================================
  Files          2716     2716           
  Lines        256117   256165   +48     
  Branches      39928    39944   +16     
=========================================
+ Hits          32499    32520   +21     
- Misses       219485   219501   +16     
- Partials       4133     4144   +11

Impacted Files	Coverage Δ
...ervisor/kvm/resource/LibvirtComputingResource.java	`18.16% <0.00%> (-0.14%)`	⬇️
...resource/wrapper/LibvirtPlugNicCommandWrapper.java	`78.04% <0.00%> (-4.01%)`	⬇️
...source/wrapper/LibvirtReplugNicCommandWrapper.java	`79.24% <0.00%> (-3.11%)`	⬇️
...ain/java/com/cloud/api/query/QueryManagerImpl.java	`2.94% <0.00%> (-0.01%)`	⬇️
...ypervisor/kvm/resource/LibvirtDomainXMLParser.java	`50.19% <33.33%> (-0.42%)`	⬇️
...pper/LibvirtPrepareForMigrationCommandWrapper.java	`43.33% <50.00%> (+0.22%)`	⬆️
...om/cloud/hypervisor/kvm/resource/LibvirtVMDef.java	`67.35% <85.71%> (+0.21%)`	⬆️

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

weizhouapache · 2023-03-15T07:53:57Z

@blueorangutan package

blueorangutan · 2023-03-15T07:55:03Z

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-03-15T08:49:16Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5723

Libvirt will automatically create the tap device. This fixes the issue with tungsten vm with multi_queue on centos7 "Unable to create tap device tap1e002c000008: Invalid argument"

weizhouapache · 2023-03-15T12:45:22Z

@blueorangutan package

blueorangutan · 2023-03-15T12:46:03Z

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-03-15T13:38:01Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5724

sonarcloud · 2023-03-15T13:39:06Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

30.2% Coverage
0.0% Duplication

weizhouapache · 2023-04-13T10:57:36Z

@blueorangutan package

blueorangutan · 2023-04-13T10:58:03Z

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

weizhouapache · 2023-04-14T11:35:16Z

@blueorangutan package

blueorangutan · 2023-04-14T11:36:03Z

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-04-14T12:33:01Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5912

borisstoyanov · 2023-05-03T09:01:33Z

@blueorangutan test keepEnv

blueorangutan · 2023-05-03T09:02:03Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

borisstoyanov · 2023-05-03T09:09:17Z

@blueorangutan package

blueorangutan · 2023-05-03T09:10:03Z

@borisstoyanov a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

borisstoyanov · 2023-05-03T10:01:53Z

@blueorangutan test keepEnv

blueorangutan · 2023-05-03T10:04:03Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-05-03T10:05:25Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6016

blueorangutan · 2023-05-03T23:14:38Z

Trillian test result (tid-6499)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45937 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7333-t6499-kvm-centos7.zip
Smoke tests completed. 109 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_10_reboot_cpvm_forced	`Error`	6.28	test_ssvm.py

borisstoyanov

LGTM, I've manually tested it and confirmed the failing test was passing in a separate run

Test force reboot CPVM ... === TestName: test_10_reboot_cpvm_forced | Status : SUCCESS ===

weizhouapache · 2023-05-04T12:45:01Z

LGTM, I've manually tested it and confirmed the failing test was passing in a separate run
Test force reboot CPVM ... === TestName: test_10_reboot_cpvm_forced | Status : SUCCESS ===

great, thanks @borisstoyanov for the testing !

rohityadavcloud

LGTM did n't test it

rohityadavcloud · 2023-05-09T09:49:04Z

Merging based on manual tests by @borisstoyanov, CI tests and reviews.

KVM: add vm setting for nic multiqueue number and packed virtqueues

877e0ce

boring-cyborg bot added component:api component:kvm component:UI labels Mar 13, 2023

weizhouapache added 3 commits March 13, 2023 12:58

kvm: add unit tests for multiqueue and packed

14170ef

KVM: set multiqueue number to the vCPU number if the value is -1

f4482a5

Tungsten: create/delete tap device with multi_queue on KVM

bcdbe30

boring-cyborg bot added the component:networking label Mar 14, 2023

Tungsten: do not create tap device before vm start

b2a1b9c

Libvirt will automatically create the tap device. This fixes the issue with tungsten vm with multi_queue on centos7 "Unable to create tap device tap1e002c000008: Invalid argument"

borisstoyanov approved these changes May 4, 2023

View reviewed changes

weizhouapache marked this pull request as ready for review May 4, 2023 12:44

rohityadavcloud added this to the 4.19.0.0 milestone May 8, 2023

rohityadavcloud approved these changes May 8, 2023

View reviewed changes

rohityadavcloud merged commit 9d46df5 into apache:main May 9, 2023

weizhouapache deleted the 4.18-kvm-support-multiqueue-and-packed branch June 23, 2023 13:10

weizhouapache mentioned this pull request Oct 2, 2023

Feature Request: KVM Multiqueue virtio-net #6904

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KVM: add vm setting for nic multiqueue number and packed virtqueues #7333

KVM: add vm setting for nic multiqueue number and packed virtqueues #7333

weizhouapache commented Mar 13, 2023

weizhouapache commented Mar 13, 2023

codecov bot commented Mar 14, 2023 •

edited

weizhouapache commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

weizhouapache commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

sonarcloud bot commented Mar 15, 2023

weizhouapache commented Apr 13, 2023

blueorangutan commented Apr 13, 2023

weizhouapache commented Apr 14, 2023

blueorangutan commented Apr 14, 2023

blueorangutan commented Apr 14, 2023

borisstoyanov commented May 3, 2023

blueorangutan commented May 3, 2023

borisstoyanov commented May 3, 2023

blueorangutan commented May 3, 2023

borisstoyanov commented May 3, 2023

blueorangutan commented May 3, 2023

blueorangutan commented May 3, 2023

blueorangutan commented May 3, 2023

borisstoyanov left a comment

weizhouapache commented May 4, 2023

rohityadavcloud left a comment

rohityadavcloud commented May 9, 2023

KVM: add vm setting for nic multiqueue number and packed virtqueues #7333

KVM: add vm setting for nic multiqueue number and packed virtqueues #7333

Conversation

weizhouapache commented Mar 13, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Screenshots (if appropriate):

weizhouapache commented Mar 13, 2023

codecov bot commented Mar 14, 2023 • edited

Codecov Report

weizhouapache commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

weizhouapache commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

blueorangutan commented Mar 15, 2023

sonarcloud bot commented Mar 15, 2023

weizhouapache commented Apr 13, 2023

blueorangutan commented Apr 13, 2023

weizhouapache commented Apr 14, 2023

blueorangutan commented Apr 14, 2023

blueorangutan commented Apr 14, 2023

borisstoyanov commented May 3, 2023

blueorangutan commented May 3, 2023

borisstoyanov commented May 3, 2023

blueorangutan commented May 3, 2023

borisstoyanov commented May 3, 2023

blueorangutan commented May 3, 2023

blueorangutan commented May 3, 2023

blueorangutan commented May 3, 2023

borisstoyanov left a comment

Choose a reason for hiding this comment

weizhouapache commented May 4, 2023

rohityadavcloud left a comment

Choose a reason for hiding this comment

rohityadavcloud commented May 9, 2023

codecov bot commented Mar 14, 2023 •

edited