KVM: Prevent regenerating keystore on provisionCertificate API #3075

nvazquez · 2018-12-02T21:59:21Z

Description

It was found out that direct download feature stops working after 'provisionCertificate' API is executed, as both features import certificates into the same /etc/cloudstack/agent/cloud.jks keystore.
The 'provisionCertificate' API invokes a script which regenerates the keystore before adding the new certificate, removing the existing certificates from the keystore. This fix prevents regenerating the keystore before adding the new certificates.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

rohityadavcloud · 2018-12-03T04:00:50Z

Can we reuse keystore table for this maybe?

...cloudstack/api/command/admin/direct/download/UploadTemplateDirectDownloadCertificateCmd.java

engine/schema/src/com/cloud/host/dao/HostDaoImpl.java

server/src/org/apache/cloudstack/direct/download/DirectDownloadManagerImpl.java

borisstoyanov · 2019-02-20T14:30:11Z

@blueorangutan package

blueorangutan · 2019-02-20T14:31:05Z

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-02-20T14:51:34Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2610

borisstoyanov · 2019-02-21T07:24:57Z

@blueorangutan test

borisstoyanov · 2019-02-25T11:45:30Z

@blueorangutan test

blueorangutan · 2019-02-25T11:46:27Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-02-25T22:33:46Z

Trillian test result (tid-3409)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37252 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3075-t3409-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Smoke tests completed. 64 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_04_extract_Iso	`Failure`	1.05	test_iso.py
test_05_stop_ssvm	`Error`	950.85	test_ssvm.py
test_06_stop_cpvm	`Error`	957.93	test_ssvm.py
test_07_reboot_ssvm	`Error`	911.62	test_ssvm.py
test_08_reboot_cpvm	`Error`	912.01	test_ssvm.py
test_04_extract_template	`Failure`	1.06	test_templates.py
test_06_download_detached_volume	`Failure`	10.27	test_volumes.py

borisstoyanov

@nvazquez there seems to be something wrong with the SSVM tests here, can you look when you have time.

nvazquez · 2019-02-26T12:07:28Z

Sure @borisstoyanov let me take a look later today

nvazquez · 2019-02-26T18:50:30Z

@blueorangutan package

blueorangutan · 2019-02-26T18:51:33Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-02-26T19:11:49Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2614

nvazquez · 2019-02-26T20:09:11Z

@borisstoyanov I have rebased the 4.11 branch and tested those files locally without errors on failing tests (results below). It seems to be errors with checksums tests.

test_ssvm.py:

Test List secondary storage VMs ... === TestName: test_01_list_sec_storage_vm | Status : SUCCESS ===
ok
Test List console proxy VMs ... === TestName: test_02_list_cpvm_vm | Status : SUCCESS ===
ok
Test SSVM Internals ... === TestName: test_03_ssvm_internals | Status : SUCCESS ===
ok
Test CPVM Internals ... === TestName: test_04_cpvm_internals | Status : SUCCESS ===
ok
Test stop SSVM ... === TestName: test_05_stop_ssvm | Status : SUCCESS ===
ok
Test stop CPVM ... === TestName: test_06_stop_cpvm | Status : SUCCESS ===
ok
Test reboot SSVM ... === TestName: test_07_reboot_ssvm | Status : SUCCESS ===
ok
Test reboot CPVM ... === TestName: test_08_reboot_cpvm | Status : SUCCESS ===
ok
Test destroy SSVM ... SKIP: Marvin configuration has no host                            credentials to check router services
Test destroy CPVM ... === TestName: test_10_destroy_cpvm | Status : SUCCESS ===
ok
Test NFS Version on Secondary Storage mounted properly on SSVM ... SKIP: No NFS version provided in test data

----------------------------------------------------------------------
Ran 11 tests in 407.304s

OK (SKIP=2)

test_iso.py, test_templates.py, test_volumes.py:

test_01_1_create_iso_with_checksum_sha1_negative (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_01_1_create_iso_with_checksum_sha1_negative | Status : SUCCESS ===
ok
test_01_create_iso_with_checksum_sha1 (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_01_create_iso_with_checksum_sha1 | Status : SUCCESS ===
ok
test_02_1_create_iso_with_checksum_sha256_negative (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_02_1_create_iso_with_checksum_sha256_negative | Status : SUCCESS ===
ok
test_02_create_iso_with_checksum_sha256 (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_02_create_iso_with_checksum_sha256 | Status : EXCEPTION ===
ERROR
=== TestName: test_02_create_iso_with_checksum_sha256 | Status : EXCEPTION ===
ERROR
test_03_1_create_iso_with_checksum_md5_negative (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_03_1_create_iso_with_checksum_md5_negative | Status : EXCEPTION ===
ERROR
test_03_create_iso_with_checksum_md5 (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_03_create_iso_with_checksum_md5 | Status : SUCCESS ===
ok
test_04_create_iso_with_no_checksum (integration.smoke.test_iso.TestCreateISOWithChecksum) ... === TestName: test_04_create_iso_with_no_checksum | Status : SUCCESS ===
ok
Test create public & private ISO ... === TestName: test_01_create_iso | Status : SUCCESS ===
ok
Test Edit ISO ... === TestName: test_02_edit_iso | Status : SUCCESS ===
ok
Test delete ISO ... === TestName: test_03_delete_iso | Status : SUCCESS ===
ok
Test for extract ISO ... === TestName: test_04_extract_Iso | Status : SUCCESS ===
ok
Update & Test for ISO permissions ... === TestName: test_05_iso_permissions | Status : SUCCESS ===
ok
Test for copy ISO from one zone to another ... SKIP: Not enough zones available to perform copy template
Test delete ISO ... === TestName: test_07_list_default_iso | Status : SUCCESS ===
ok
test_09_copy_delete_template (integration.smoke.test_templates.TestCopyDeleteTemplate) ... SKIP: Skipping test due to there are less than two zones.
Test create public & private template ... === TestName: test_01_create_template | Status : SUCCESS ===
ok
Test when createTemplate is used to create templates having the same name all of them get ... === TestName: test_CreateTemplateWithDuplicateName | Status : SUCCESS ===
ok
test_02_1_create_template_with_checksum_sha1_negative (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_02_1_create_template_with_checksum_sha1_negative | Status : SUCCESS ===
ok
test_02_create_template_with_checksum_sha1 (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_02_create_template_with_checksum_sha1 | Status : EXCEPTION ===
ERROR
test_03_1_create_template_with_checksum_sha256_negative (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_03_1_create_template_with_checksum_sha256_negative | Status : SUCCESS ===
ok
test_03_create_template_with_checksum_sha256 (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_03_create_template_with_checksum_sha256 | Status : SUCCESS ===
ok
test_04_1_create_template_with_checksum_md5_negative (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_04_1_create_template_with_checksum_md5_negative | Status : SUCCESS ===
ok
test_04_create_template_with_checksum_md5 (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_04_create_template_with_checksum_md5 | Status : SUCCESS ===
ok
test_05_create_template_with_no_checksum (integration.smoke.test_templates.TestCreateTemplateWithChecksum) ... === TestName: test_05_create_template_with_no_checksum | Status : SUCCESS ===
ok
Register a template using Direct Download flag ... === TestName: test_01_register_template_direct_download_flag | Status : SUCCESS ===
ok
Deploy a VM from a Direct Download registered template ... === TestName: test_02_deploy_vm_from_direct_download_template | Status : SUCCESS ===
ok
Deploy a VM from a Direct Download registered template with wrong checksum ... === TestName: test_03_deploy_vm_wrong_checksum | Status : SUCCESS ===
ok
Test Edit template ... === TestName: test_02_edit_template | Status : SUCCESS ===
ok
Test delete template ... === TestName: test_03_delete_template | Status : SUCCESS ===
ok
Test for extract template ... === TestName: test_04_extract_template | Status : SUCCESS ===
ok
Update & Test for template permissions ... === TestName: test_05_template_permissions | Status : SUCCESS ===
ok
Test for copy template from one zone to another ... SKIP: Not enough zones available to perform copy template
Test only public templates are visible to normal user ... === TestName: test_07_list_public_templates | Status : SUCCESS ===
ok
Test System templates are not visible to normal user ... === TestName: test_08_list_system_templates | Status : SUCCESS ===
ok
Test Volume creation for all Disk Offerings (incl. custom) ... === TestName: test_01_create_volume | Status : SUCCESS ===
ok
Attach a created Volume to a Running VM ... === TestName: test_02_attach_volume | Status : SUCCESS ===
ok
Download a Volume attached to a VM ... === TestName: test_03_download_attached_volume | Status : SUCCESS ===
ok
Delete a Volume attached to a VM ... === TestName: test_04_delete_attached_volume | Status : SUCCESS ===
ok
Detach a Volume attached to a VM ... === TestName: test_05_detach_volume | Status : SUCCESS ===
ok
Download a Volume unattached to an VM ... === TestName: test_06_download_detached_volume | Status : SUCCESS ===
ok
Test resize (negative) non-existent volume ... === TestName: test_07_resize_fail | Status : SUCCESS ===
ok
Test resize a volume ... === TestName: test_08_resize_volume | Status : SUCCESS ===
ok
Delete a Volume unattached to an VM ... === TestName: test_09_delete_detached_volume | Status : SUCCESS ===
ok
test_10_list_volumes (integration.smoke.test_volumes.TestVolumes) ... === TestName: test_10_list_volumes | Status : SUCCESS ===
ok

blueorangutan · 2019-02-26T20:09:33Z

@nvazquez unsupported parameters provided. Supported mgmt server os are: centos6, centos7, ubuntu. Supported hypervisors are: kvm-centos6, kvm-centos7, kvm-ubuntu, xenserver-71, xenserver-65sp1, xenserver-62sp1, vmware-65, vmware-60u2, vmware-55u3, vmware-51u1, vmware-50u1

nvazquez · 2019-02-26T20:12:11Z

@blueorangutan test

blueorangutan · 2019-02-26T20:12:33Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-02-27T07:05:38Z

Trillian test result (tid-3410)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37431 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3075-t3410-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 64 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_04_extract_Iso	`Failure`	1.08	test_iso.py
test_05_stop_ssvm	`Error`	941.19	test_ssvm.py
test_06_stop_cpvm	`Error`	960.49	test_ssvm.py
test_07_reboot_ssvm	`Error`	912.87	test_ssvm.py
test_08_reboot_cpvm	`Error`	912.91	test_ssvm.py
test_05_create_template_with_no_checksum	`Error`	65.50	test_templates.py
test_04_extract_template	`Failure`	1.09	test_templates.py
test_06_download_detached_volume	`Failure`	23.77	test_volumes.py

rohityadavcloud · 2019-05-30T06:18:44Z

@blueorangutan package

blueorangutan · 2019-06-04T19:42:22Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2843

nvazquez · 2019-06-04T19:46:50Z

@blueorangutan test

blueorangutan · 2019-06-04T19:47:51Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

nvazquez · 2019-06-04T19:49:20Z

@rhtyd this is ready for review

rohityadavcloud · 2019-06-05T03:20:15Z

scripts/util/keystore-cert-import

@@ -38,9 +38,6 @@ if [ -z "${KS_PASS// }" ]; then
    exit 1
 fi

-# Use a new keystore file
-NEW_KS_FILE="$KS_FILE.new"


The script was create to be idempotent for jks creation and cert import. I'll have to test concurrent cases and see if this script would pass/fail. I'll test and review today and keep you posted @nvazquez thanks.

I'll also need to test for certificate renewal.

blueorangutan · 2019-06-05T07:50:50Z

Trillian test result (tid-3645)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41854 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3075-t3645-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 68 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_05_stop_ssvm	`Error`	961.34	test_ssvm.py
test_06_stop_cpvm	`Error`	955.20	test_ssvm.py
test_07_reboot_ssvm	`Error`	911.84	test_ssvm.py
test_08_reboot_cpvm	`Error`	910.82	test_ssvm.py

scripts/util/keystore-setup

…ficates

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud · 2019-06-05T10:55:42Z

@nvazquez LGTM, I reviewed and fixed few issues around certificate renewal
@blueorangutan package

blueorangutan · 2019-06-05T10:55:53Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-06-05T11:16:16Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2847

rohityadavcloud · 2019-06-05T11:19:30Z

@blueorangutan test

blueorangutan · 2019-06-05T11:19:54Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

DaanHoogland

code changes look good, testing this is done and i'll have to trust that but it is scary ;)

blueorangutan · 2019-06-05T19:28:46Z

Trillian test result (tid-3650)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 27669 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3075-t3650-kvm-centos7.zip
Smoke tests completed. 69 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

nvazquez · 2019-06-05T20:29:21Z

Thanks for the last fix @rhtyd
Merging after test results and reviews

nvazquez requested review from rohityadavcloud, rafaelweingartner and DaanHoogland December 2, 2018 21:59

nvazquez added this to the 4.11.3.0 milestone Dec 2, 2018

nvazquez mentioned this pull request Dec 2, 2018

KVM: Improvements on upload direct download certificates #2995

Merged

5 tasks

rafaelweingartner reviewed Dec 12, 2018

View reviewed changes

nvazquez changed the title ~~Prevent regenerating keystore and add background task to upload certificates~~ WIP: Prevent regenerating keystore and add background task to upload certificates Feb 25, 2019

borisstoyanov requested changes Feb 26, 2019

View reviewed changes

nvazquez force-pushed the directdownloadkeystoreissue branch from 322ca1a to 35fa34d Compare February 26, 2019 18:29

rohityadavcloud added the status:work-in-progress label Mar 25, 2019

apache deleted a comment from blueorangutan May 30, 2019

nvazquez removed the component:templates label Jun 4, 2019

nvazquez requested review from rafaelweingartner and borisstoyanov June 4, 2019 19:50

nvazquez mentioned this pull request Jun 5, 2019

KVM: Enhancements for direct download feature #3374

Merged

5 tasks

rohityadavcloud reviewed Jun 5, 2019

View reviewed changes

apache deleted a comment from blueorangutan Jun 5, 2019

rohityadavcloud reviewed Jun 5, 2019

View reviewed changes

scripts/util/keystore-setup Outdated Show resolved Hide resolved

nvazquez added 2 commits June 5, 2019 14:45

Prevent regenerating keystore and add background task to upload certi…

b7e99a3

…ficates

Remove background task to upload certificates

07a550c

rohityadavcloud force-pushed the directdownloadkeystoreissue branch from 4c9bca6 to 07a550c Compare June 5, 2019 09:15

rohityadavcloud added 2 commits June 5, 2019 16:03

keystore-setup: delete old key in jks

e1d8555

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

also don't allow ca-cert alias

7831806

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud approved these changes Jun 5, 2019

View reviewed changes

rohityadavcloud requested a review from PaulAngus June 5, 2019 10:59

DaanHoogland approved these changes Jun 5, 2019

View reviewed changes

nvazquez merged commit d108f32 into apache:4.11 Jun 5, 2019

nvazquez deleted the directdownloadkeystoreissue branch April 6, 2020 14:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KVM: Prevent regenerating keystore on provisionCertificate API #3075

KVM: Prevent regenerating keystore on provisionCertificate API #3075

nvazquez commented Dec 2, 2018 •

edited

rohityadavcloud commented Dec 3, 2018

borisstoyanov commented Feb 20, 2019

blueorangutan commented Feb 20, 2019

blueorangutan commented Feb 20, 2019

borisstoyanov commented Feb 21, 2019

borisstoyanov commented Feb 25, 2019

blueorangutan commented Feb 25, 2019

blueorangutan commented Feb 25, 2019

borisstoyanov left a comment

nvazquez commented Feb 26, 2019

nvazquez commented Feb 26, 2019

blueorangutan commented Feb 26, 2019

blueorangutan commented Feb 26, 2019

nvazquez commented Feb 26, 2019 •

edited

blueorangutan commented Feb 26, 2019

nvazquez commented Feb 26, 2019

blueorangutan commented Feb 26, 2019

blueorangutan commented Feb 27, 2019

rohityadavcloud commented May 30, 2019

blueorangutan commented Jun 4, 2019

nvazquez commented Jun 4, 2019

blueorangutan commented Jun 4, 2019

nvazquez commented Jun 4, 2019

rohityadavcloud Jun 5, 2019

rohityadavcloud Jun 5, 2019

blueorangutan commented Jun 5, 2019

rohityadavcloud commented Jun 5, 2019

blueorangutan commented Jun 5, 2019

blueorangutan commented Jun 5, 2019

rohityadavcloud commented Jun 5, 2019

blueorangutan commented Jun 5, 2019

DaanHoogland left a comment

blueorangutan commented Jun 5, 2019

nvazquez commented Jun 5, 2019

KVM: Prevent regenerating keystore on provisionCertificate API #3075

KVM: Prevent regenerating keystore on provisionCertificate API #3075

Conversation

nvazquez commented Dec 2, 2018 • edited

Description

Types of changes

Screenshots (if appropriate):

How Has This Been Tested?

rohityadavcloud commented Dec 3, 2018

borisstoyanov commented Feb 20, 2019

blueorangutan commented Feb 20, 2019

blueorangutan commented Feb 20, 2019

borisstoyanov commented Feb 21, 2019

borisstoyanov commented Feb 25, 2019

blueorangutan commented Feb 25, 2019

blueorangutan commented Feb 25, 2019

borisstoyanov left a comment

Choose a reason for hiding this comment

nvazquez commented Feb 26, 2019

nvazquez commented Feb 26, 2019

blueorangutan commented Feb 26, 2019

blueorangutan commented Feb 26, 2019

nvazquez commented Feb 26, 2019 • edited

blueorangutan commented Feb 26, 2019

nvazquez commented Feb 26, 2019

blueorangutan commented Feb 26, 2019

blueorangutan commented Feb 27, 2019

rohityadavcloud commented May 30, 2019

blueorangutan commented Jun 4, 2019

nvazquez commented Jun 4, 2019

blueorangutan commented Jun 4, 2019

nvazquez commented Jun 4, 2019

rohityadavcloud Jun 5, 2019

Choose a reason for hiding this comment

rohityadavcloud Jun 5, 2019

Choose a reason for hiding this comment

blueorangutan commented Jun 5, 2019

rohityadavcloud commented Jun 5, 2019

blueorangutan commented Jun 5, 2019

blueorangutan commented Jun 5, 2019

rohityadavcloud commented Jun 5, 2019

blueorangutan commented Jun 5, 2019

DaanHoogland left a comment

Choose a reason for hiding this comment

blueorangutan commented Jun 5, 2019

nvazquez commented Jun 5, 2019

nvazquez commented Dec 2, 2018 •

edited

nvazquez commented Feb 26, 2019 •

edited