Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable DPDK support on KVM #2839

Merged
merged 3 commits into from Nov 7, 2018

Conversation

@nvazquez
Copy link
Contributor

commented Sep 9, 2018

Description

Enable DPDK support on KVM in conjunction with Open VSwitch.

New properties can be set on agent.properties file to enable ports created with dpdkvhostuser type on KVM hosts:

openvswitch.dpdk.enabled=true
openvswitch.dpdk.ovs.path=OVS_PATH

DPDK guest VMs can be set by sending extra configurations as part of the 'deployVirtualMachine' or 'updateVirtualMachine' API methods. Extra configurations are controlled by a new global setting 'enable.additional.vm.configuration'.
Extra configurations are parts of XML data which are appended to the domail XML of the VM. CloudStack expects an UTF-8 URL encoded string as 'extraconfig' parameter on the mentioned API methods.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

GitHub Issue/PRs

Documentation PR: apache/cloudstack-documentation#6 & apache/cloudstack-documentation#8

Screenshots (if appropriate):

How Has This Been Tested?

Checklist:

  • I have read the CONTRIBUTING document.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
    Testing
  • I have added tests to cover my changes.
  • All relevant new and existing integration tests have passed.
  • A full integration testsuite with all test that can run on my environment has passed.
@borisstoyanov

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2018

@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 10, 2018

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 10, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2292

@borisstoyanov

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 10, 2018

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 10, 2018

Trillian test result (tid-2996)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28491 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t2996-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 66 look OK, 3 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_03_vpc_privategw_restart_vpc_cleanup Failure 1162.08 test_privategw_acl.py
test_05_rvpc_multi_tiers Failure 334.10 test_vpc_redundant.py
test_05_rvpc_multi_tiers Error 359.25 test_vpc_redundant.py
test_hostha_enable_ha_when_host_in_maintenance Error 2.60 test_hostha_kvm.py
@dhlaluku
Copy link
Contributor

left a comment

Nice work Nicolas, LGTM

@borisstoyanov
Copy link
Contributor

left a comment

LGTM

@resmo resmo added the component:kvm label Sep 17, 2018

@nvazquez nvazquez added this to the 4.12.0.0 milestone Sep 17, 2018

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Sep 17, 2018

@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 17, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 17, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2299

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Sep 17, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 17, 2018

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@borisstoyanov
Copy link
Contributor

left a comment

@nvazquez can you please add some documentation and references how to use this feature, otherwise LGTM.

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 18, 2018

Trillian test result (tid-3019)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33057 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3019-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 65 look OK, 4 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_nic_secondaryip_add_remove Error 42.04 test_multipleips_per_nic.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 1240.65 test_privategw_acl.py
test_03_secured_to_nonsecured_vm_migration Error 53.99 test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration Error 44.23 test_vm_life_cycle.py
test_02_cancel_host_maintenace_with_migration_jobs Error 2.38 test_host_maintenance.py

@nvazquez nvazquez force-pushed the shapeblue:dpdksupport-master branch 2 times, most recently from eb79cf1 to 894840f Sep 20, 2018

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Sep 20, 2018

Thanks @borisstoyanov, I've updated the PR description and also rebased master branch
@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 20, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 20, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2310

@borisstoyanov

This comment has been minimized.

Copy link
Contributor

commented Sep 20, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 20, 2018

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

This comment has been minimized.

Copy link

commented Sep 20, 2018

Trillian test result (tid-3033)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 23866 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3033-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 62 look OK, 7 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_nic_secondaryip_add_remove Error 25.10 test_multipleips_per_nic.py
test_01_add_primary_storage_disabled_host Error 0.52 test_primary_storage.py
test_01_primary_storage_nfs Error 0.09 test_primary_storage.py
ContextSuite context=TestStorageTags>:setup Error 0.16 test_primary_storage.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 1098.05 test_privategw_acl.py
test_02_list_snapshots_with_removed_data_store Error 1.13 test_snapshots.py
test_01_secure_vm_migration Error 75.38 test_vm_life_cycle.py
test_02_unsecure_vm_migration Error 92.56 test_vm_life_cycle.py
test_08_migrate_vm Error 14.62 test_vm_life_cycle.py
test_01_cancel_host_maintenace_with_no_migration_jobs Failure 0.08 test_host_maintenance.py
test_02_cancel_host_maintenace_with_migration_jobs Error 2.22 test_host_maintenance.py
test_hostha_enable_ha_when_host_in_maintenance Error 2.47 test_hostha_kvm.py
test_hostha_kvm_host_fencing Failure 631.63 test_hostha_kvm.py
test_hostha_kvm_host_recovering Failure 614.20 test_hostha_kvm.py

@nvazquez nvazquez force-pushed the shapeblue:dpdksupport-master branch from 894840f to aa0e29d Sep 21, 2018

@nvazquez nvazquez force-pushed the shapeblue:dpdksupport-master branch from aa0e29d to e4438cd Oct 1, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Oct 7, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Oct 7, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2334

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Oct 7, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Oct 7, 2018

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@rafaelweingartner rafaelweingartner force-pushed the apache:master branch from 1fa4f10 to fcc87d9 Oct 7, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Oct 8, 2018

Trillian test result (tid-3062)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 25057 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3062-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 64 look OK, 5 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_nic_secondaryip_add_remove Error 25.11 test_multipleips_per_nic.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 1107.80 test_privategw_acl.py
test_02_unsecure_vm_migration Error 130.98 test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration Error 75.52 test_vm_life_cycle.py
test_04_rvpc_network_garbage_collector_nics Failure 471.46 test_vpc_redundant.py
test_hostha_enable_ha_when_host_in_maintenance Error 2.42 test_hostha_kvm.py
@borisstoyanov
Copy link
Contributor

left a comment

LGTM, test failures does not seem related

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Oct 8, 2018

Thanks @borisstoyanov, I'm seeing Jenkins failures to get the code and start checking and tried rebuilding the job but still fails with:

ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/apache/cloudstack
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
	at hudson.scm.SCM.checkout(SCM.java:504)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1794)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git reset --hard" returned status code 128:
stdout: 
stderr: fatal: unable to read tree b4862e12b63821371306b72302066bf6772d774b

@nvazquez nvazquez force-pushed the shapeblue:dpdksupport-master branch from 609dd55 to 53daf8f Oct 10, 2018

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Oct 10, 2018

Rebased master branch again but still getting Jenkins error trying to get the code @rhtyd @borisstoyanov

@@ -514,6 +515,8 @@
private static final ConfigKey<Boolean> AllowDeployVmIfGivenHostFails = new ConfigKey<Boolean>("Advanced", Boolean.class, "allow.deploy.vm.if.deploy.on.given.host.fails", "false",
"allow vm to deploy on different host if vm fails to deploy on the given host ", true);

private static final ConfigKey<Boolean> EnableAdditionalVmConfig = new ConfigKey<>("Advanced", Boolean.class, "enable.additional.vm.configuration",

This comment has been minimized.

Copy link
@dhlaluku

dhlaluku Oct 11, 2018

Contributor

@nvazquez Can we change this setting from 'enable.additional.vm.configuration' to 'additional.configuration.vm.allow' or 'additional.configuration.vm.enable'?

@rhtyd

This comment has been minimized.

Copy link
Member

commented Oct 25, 2018

@nvazquez is this still in progress?

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Oct 25, 2018

@rhtyd yes, I'll update it with latest fix once testing is completed

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 5, 2018

@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 5, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2405

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 6, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

Trillian test result (tid-3143)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 21856 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2839-t3143-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 67 look OK, 2 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_nic_secondaryip_add_remove Error 38.31 test_multipleips_per_nic.py
test_04_rvpc_network_garbage_collector_nics Failure 437.03 test_vpc_redundant.py
@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 6, 2018

@DaanHoogland @borisstoyanov latest fixes added, are we ok to merge this?

@DaanHoogland

This comment has been minimized.

Copy link
Contributor

commented Nov 6, 2018

think we are @nvazquez

@borisstoyanov
Copy link
Contributor

left a comment

yeah LGTM

@nvazquez nvazquez merged commit 4de4eab into apache:master Nov 7, 2018

2 checks passed

Jenkins This pull request looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@nvazquez nvazquez deleted the shapeblue:dpdksupport-master branch Nov 7, 2018

@nvazquez nvazquez referenced this pull request Jan 28, 2019
1 of 5 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.