Enable DPDK support on KVM #2839

nvazquez · 2018-09-09T21:09:20Z

Description

Enable DPDK support on KVM in conjunction with Open VSwitch.

New properties can be set on agent.properties file to enable ports created with dpdkvhostuser type on KVM hosts:

openvswitch.dpdk.enabled=true
openvswitch.dpdk.ovs.path=OVS_PATH

DPDK guest VMs can be set by sending extra configurations as part of the 'deployVirtualMachine' or 'updateVirtualMachine' API methods. Extra configurations are controlled by a new global setting 'enable.additional.vm.configuration'.
Extra configurations are parts of XML data which are appended to the domail XML of the VM. CloudStack expects an UTF-8 URL encoded string as 'extraconfig' parameter on the mentioned API methods.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

GitHub Issue/PRs

Documentation PR: apache/cloudstack-documentation#6 & apache/cloudstack-documentation#8

Screenshots (if appropriate):

How Has This Been Tested?

Checklist:

I have read the CONTRIBUTING document.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
Testing
I have added tests to cover my changes.
All relevant new and existing integration tests have passed.
A full integration testsuite with all test that can run on my environment has passed.

borisstoyanov · 2018-09-10T10:59:27Z

@blueorangutan package

blueorangutan · 2018-09-10T11:00:30Z

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2018-09-10T11:27:33Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2292

borisstoyanov · 2018-09-10T11:47:36Z

@blueorangutan test

blueorangutan · 2018-09-10T11:48:31Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2018-09-10T20:17:37Z

Trillian test result (tid-2996)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28491 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t2996-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 66 look OK, 3 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	1162.08	test_privategw_acl.py
test_05_rvpc_multi_tiers	`Failure`	334.10	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	359.25	test_vpc_redundant.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	2.60	test_hostha_kvm.py

dhlaluku

Nice work Nicolas, LGTM

borisstoyanov

LGTM

nvazquez · 2018-09-17T18:29:38Z

@blueorangutan package

blueorangutan · 2018-09-17T18:29:51Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2018-09-17T19:09:01Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2299

nvazquez · 2018-09-17T20:17:58Z

@blueorangutan test

blueorangutan · 2018-09-17T20:18:51Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

borisstoyanov

@nvazquez can you please add some documentation and references how to use this feature, otherwise LGTM.

blueorangutan · 2018-09-18T20:39:15Z

Trillian test result (tid-3019)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33057 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3019-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 65 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_nic_secondaryip_add_remove	`Error`	42.04	test_multipleips_per_nic.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	1240.65	test_privategw_acl.py
test_03_secured_to_nonsecured_vm_migration	`Error`	53.99	test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration	`Error`	44.23	test_vm_life_cycle.py
test_02_cancel_host_maintenace_with_migration_jobs	`Error`	2.38	test_host_maintenance.py

nvazquez · 2018-09-20T03:54:53Z

Thanks @borisstoyanov, I've updated the PR description and also rebased master branch
@blueorangutan package

blueorangutan · 2018-09-20T03:56:03Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2018-09-20T04:29:39Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2310

borisstoyanov · 2018-09-20T06:11:09Z

@blueorangutan test

blueorangutan · 2018-09-20T06:12:02Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2018-09-20T13:19:06Z

Trillian test result (tid-3033)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 23866 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3033-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 62 look OK, 7 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_nic_secondaryip_add_remove	`Error`	25.10	test_multipleips_per_nic.py
test_01_add_primary_storage_disabled_host	`Error`	0.52	test_primary_storage.py
test_01_primary_storage_nfs	`Error`	0.09	test_primary_storage.py
ContextSuite context=TestStorageTags>:setup	`Error`	0.16	test_primary_storage.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	1098.05	test_privategw_acl.py
test_02_list_snapshots_with_removed_data_store	`Error`	1.13	test_snapshots.py
test_01_secure_vm_migration	`Error`	75.38	test_vm_life_cycle.py
test_02_unsecure_vm_migration	`Error`	92.56	test_vm_life_cycle.py
test_08_migrate_vm	`Error`	14.62	test_vm_life_cycle.py
test_01_cancel_host_maintenace_with_no_migration_jobs	`Failure`	0.08	test_host_maintenance.py
test_02_cancel_host_maintenace_with_migration_jobs	`Error`	2.22	test_host_maintenance.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	2.47	test_hostha_kvm.py
test_hostha_kvm_host_fencing	`Failure`	631.63	test_hostha_kvm.py
test_hostha_kvm_host_recovering	`Failure`	614.20	test_hostha_kvm.py

nvazquez · 2018-10-07T16:31:56Z

Rebased master branch.
@blueorangutan package

blueorangutan · 2018-10-07T16:32:21Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2018-10-07T17:05:30Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2334

nvazquez · 2018-10-07T17:21:57Z

@blueorangutan test

blueorangutan · 2018-10-07T17:22:21Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2018-10-08T00:49:46Z

Trillian test result (tid-3062)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 25057 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3062-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 64 look OK, 5 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_nic_secondaryip_add_remove	`Error`	25.11	test_multipleips_per_nic.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	1107.80	test_privategw_acl.py
test_02_unsecure_vm_migration	`Error`	130.98	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	`Error`	75.52	test_vm_life_cycle.py
test_04_rvpc_network_garbage_collector_nics	`Failure`	471.46	test_vpc_redundant.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	2.42	test_hostha_kvm.py

borisstoyanov

LGTM, test failures does not seem related

nvazquez · 2018-10-08T14:09:27Z

Thanks @borisstoyanov, I'm seeing Jenkins failures to get the code and start checking and tried rebuilding the job but still fails with:

ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/apache/cloudstack
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
	at hudson.scm.SCM.checkout(SCM.java:504)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1794)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git reset --hard" returned status code 128:
stdout: 
stderr: fatal: unable to read tree b4862e12b63821371306b72302066bf6772d774b

nvazquez · 2018-10-10T18:20:08Z

Rebased master branch again but still getting Jenkins error trying to get the code @rhtyd @borisstoyanov

dhlaluku · 2018-10-11T09:48:39Z

server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

@@ -514,6 +515,8 @@
    private static final ConfigKey<Boolean> AllowDeployVmIfGivenHostFails = new ConfigKey<Boolean>("Advanced", Boolean.class, "allow.deploy.vm.if.deploy.on.given.host.fails", "false",
            "allow vm to deploy on different host if vm fails to deploy on the given host ", true);

+    private static final ConfigKey<Boolean> EnableAdditionalVmConfig = new ConfigKey<>("Advanced", Boolean.class, "enable.additional.vm.configuration",


@nvazquez Can we change this setting from 'enable.additional.vm.configuration' to 'additional.configuration.vm.allow' or 'additional.configuration.vm.enable'?

rohityadavcloud · 2018-10-25T01:28:57Z

@nvazquez is this still in progress?

nvazquez · 2018-10-25T03:31:06Z

@rhtyd yes, I'll update it with latest fix once testing is completed

nvazquez · 2018-11-05T23:35:59Z

@blueorangutan package

blueorangutan · 2018-11-05T23:36:49Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2018-11-06T00:09:58Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2405

nvazquez · 2018-11-06T00:17:14Z

@blueorangutan test

blueorangutan · 2018-11-06T00:17:49Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2018-11-06T06:47:36Z

Trillian test result (tid-3143)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 21856 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3143-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 67 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_nic_secondaryip_add_remove	`Error`	38.31	test_multipleips_per_nic.py
test_04_rvpc_network_garbage_collector_nics	`Failure`	437.03	test_vpc_redundant.py

nvazquez · 2018-11-06T14:23:52Z

@DaanHoogland @borisstoyanov latest fixes added, are we ok to merge this?

DaanHoogland · 2018-11-06T15:57:22Z

think we are @nvazquez

borisstoyanov

yeah LGTM

dhlaluku approved these changes Sep 12, 2018

View reviewed changes

borisstoyanov approved these changes Sep 14, 2018

View reviewed changes

resmo added the component:kvm label Sep 17, 2018

nvazquez added the component:openvswitch label Sep 17, 2018

nvazquez added this to the 4.12.0.0 milestone Sep 17, 2018

borisstoyanov reviewed Sep 18, 2018

View reviewed changes

nvazquez force-pushed the dpdksupport-master branch 2 times, most recently from eb79cf1 to 894840f Compare September 20, 2018 03:47

DaanHoogland approved these changes Sep 21, 2018

View reviewed changes

nvazquez force-pushed the dpdksupport-master branch from 894840f to aa0e29d Compare September 21, 2018 14:46

nvazquez added the status:work-in-progress label Sep 21, 2018

nvazquez force-pushed the dpdksupport-master branch from aa0e29d to e4438cd Compare October 1, 2018 01:47

rafaelweingartner force-pushed the master branch from 1fa4f10 to fcc87d9 Compare October 7, 2018 21:14

borisstoyanov approved these changes Oct 8, 2018

View reviewed changes

Enable DPDK support on KVM

53daf8f

nvazquez force-pushed the dpdksupport-master branch from 609dd55 to 53daf8f Compare October 10, 2018 14:36

dhlaluku reviewed Oct 11, 2018

View reviewed changes

nvazquez added 2 commits October 29, 2018 12:47

Allow DPDK deployments on user VMs only

ef37bb3

Fix port name ordering

26058e3

nvazquez removed the status:work-in-progress label Nov 5, 2018

borisstoyanov approved these changes Nov 6, 2018

View reviewed changes

nvazquez merged commit 4de4eab into apache:master Nov 7, 2018

nvazquez deleted the dpdksupport-master branch November 7, 2018 12:30

nvazquez mentioned this pull request Jan 28, 2019

DPDK vHost User mode selection #3153

Merged

5 tasks

Enable DPDK support on KVM #2839

Enable DPDK support on KVM #2839

Conversation

nvazquez commented Sep 9, 2018 • edited by andrijapanicsb Loading

Description

Types of changes

GitHub Issue/PRs

Screenshots (if appropriate):

How Has This Been Tested?

Checklist:

borisstoyanov commented Sep 10, 2018

blueorangutan commented Sep 10, 2018

blueorangutan commented Sep 10, 2018

borisstoyanov commented Sep 10, 2018

blueorangutan commented Sep 10, 2018

blueorangutan commented Sep 10, 2018

dhlaluku left a comment

Choose a reason for hiding this comment

borisstoyanov left a comment

Choose a reason for hiding this comment

nvazquez commented Sep 17, 2018

blueorangutan commented Sep 17, 2018

blueorangutan commented Sep 17, 2018

nvazquez commented Sep 17, 2018

blueorangutan commented Sep 17, 2018

borisstoyanov left a comment

Choose a reason for hiding this comment

blueorangutan commented Sep 18, 2018

nvazquez commented Sep 20, 2018

blueorangutan commented Sep 20, 2018

blueorangutan commented Sep 20, 2018

borisstoyanov commented Sep 20, 2018

blueorangutan commented Sep 20, 2018

blueorangutan commented Sep 20, 2018

nvazquez commented Oct 7, 2018

blueorangutan commented Oct 7, 2018

blueorangutan commented Oct 7, 2018

nvazquez commented Oct 7, 2018

blueorangutan commented Oct 7, 2018

blueorangutan commented Oct 8, 2018

borisstoyanov left a comment

Choose a reason for hiding this comment

nvazquez commented Oct 8, 2018

nvazquez commented Oct 10, 2018

dhlaluku Oct 11, 2018 • edited Loading

Choose a reason for hiding this comment

rohityadavcloud commented Oct 25, 2018

nvazquez commented Oct 25, 2018

nvazquez commented Nov 5, 2018

blueorangutan commented Nov 5, 2018

blueorangutan commented Nov 6, 2018

nvazquez commented Nov 6, 2018

blueorangutan commented Nov 6, 2018

blueorangutan commented Nov 6, 2018

nvazquez commented Nov 6, 2018

DaanHoogland commented Nov 6, 2018

borisstoyanov left a comment

Choose a reason for hiding this comment

nvazquez commented Sep 9, 2018 •

edited by andrijapanicsb

Loading

dhlaluku Oct 11, 2018 •

edited

Loading