Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CLOUDSTACK-10153: Introduce string API arg trust validation #2336

Closed
wants to merge 2 commits into from
Closed

CLOUDSTACK-10153: Introduce string API arg trust validation #2336

wants to merge 2 commits into from

Conversation

rohityadavcloud
Copy link
Member

Adds new API string argument/field validator that validates the string
as untrusted HTML using owasp's java-html-sanitizer. This also adds
a API argument validator type to skip validations, useful for fields
such as certificates and keys.

Signed-off-by: Rohit Yadav rohit.yadav@shapeblue.com

@borisstoyanov
Copy link
Contributor

Thanks for this enhancement @rhtyd
@blueorangutan package

@blueorangutan
Copy link

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1287

@borisstoyanov
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@rohityadavcloud
Copy link
Member Author

@borisstoyanov looks like there is something really wrong with the changes, let me test as all the travis jobs failed.

@rohityadavcloud
CLOUDSTACK-10153: Introduce string API arg trust validation
9705a97 
Adds new API string argument/field validator that validates the string
as untrusted HTML using owasp's java-html-sanitizer. This also adds
a API argument validator type to skip validations, useful for fields
such as certificates and keys.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
@rohityadavcloud
switch to jsoup to clean possible html string
bea1198 
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
@rohityadavcloud
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1290

@rohityadavcloud
Copy link
Member Author

@blueorangutan test

@blueorangutan
Copy link

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

Trillian test result (tid-1690)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 51556 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2336-t1690-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_host_annotations.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_ssvm.py
Intermitten failure detected: /marvin/tests/smoke/test_templates.py
Intermitten failure detected: /marvin/tests/smoke/test_usage.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermitten failure detected: /marvin/tests/smoke/test_volumes.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 58 look OK, 9 have error(s)

Test Result Time (s) Test File
test_07_resize_fail Failure 15.36 test_volumes.py
test_10_attachAndDetach_iso Failure 1513.76 test_vm_life_cycle.py
test_09_destroy_ssvm Failure 0.03 test_ssvm.py
test_08_reboot_cpvm Failure 0.03 test_ssvm.py
test_07_reboot_ssvm Failure 0.04 test_ssvm.py
test_06_stop_cpvm Failure 0.04 test_ssvm.py
test_05_stop_ssvm Failure 0.03 test_ssvm.py
test_04_cpvm_internals Failure 0.03 test_ssvm.py
test_03_ssvm_internals Failure 0.03 test_ssvm.py
test_02_list_cpvm_vm Failure 0.03 test_ssvm.py
test_01_list_sec_storage_vm Failure 0.04 test_ssvm.py
test_04_rvpc_privategw_static_routes Failure 213.38 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 142.87 test_privategw_acl.py
test_02_vpc_privategw_static_routes Failure 102.76 test_privategw_acl.py
test_01_vpc_privategw_acl Failure 41.56 test_privategw_acl.py
ContextSuite context=TestVpcSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVPCRedundancy>:setup Error 0.00 test_vpc_redundant.py
test_06_download_detached_volume Error 20.33 test_volumes.py
test_01_volume_usage Error 196.30 test_usage.py
test_01_template_usage Error 80.79 test_usage.py
ContextSuite context=TestISOUsage>:setup Error 0.00 test_usage.py
test_05_create_template_with_no_checksum Error 65.64 test_templates.py
test_04_extract_template Error 5.08 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.67 test_templates.py
test_03_delete_template Error 5.10 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.66 test_templates.py
test_02_create_template_with_checksum_sha1 Error 65.76 test_templates.py
test_01_create_template Error 35.39 test_templates.py
test_10_destroy_cpvm Error 5.15 test_ssvm.py
test_05_add_annotation_for_invalid_entityType Error 0.10 test_host_annotations.py
test_change_service_offering_for_vm_with_snapshots Skipped 0.00 test_vm_snapshots.py
test_09_copy_delete_template Skipped 0.02 test_templates.py
test_06_copy_template Skipped 0.00 test_templates.py
test_static_role_account_acls Skipped 0.02 test_staticroles.py
test_11_ss_nfs_version_on_ssvm Skipped 0.03 test_ssvm.py
test_01_scale_vm Skipped 0.00 test_scale_vm.py
test_01_primary_storage_iscsi Skipped 0.12 test_primary_storage.py
test_vm_nic_adapter_vmxnet3 Skipped 0.00 test_nic_adapter_type.py
test_03_nic_multiple_vmware Skipped 1.17 test_nic.py
test_nested_virtualization_vmware Skipped 0.00 test_nested_virtualization.py
test_06_copy_iso Skipped 0.00 test_iso.py
test_list_ha_for_host_valid Skipped 0.03 test_hostha_simulator.py
test_list_ha_for_host_invalid Skipped 0.02 test_hostha_simulator.py
test_list_ha_for_host Skipped 0.02 test_hostha_simulator.py
test_hostha_enable_feature_without_setting_provider Skipped 0.02 test_hostha_simulator.py
test_hostha_enable_feature_valid Skipped 0.02 test_hostha_simulator.py
test_hostha_disable_feature_valid Skipped 0.02 test_hostha_simulator.py
test_hostha_configure_invalid_provider Skipped 0.02 test_hostha_simulator.py
test_hostha_configure_default_driver Skipped 0.02 test_hostha_simulator.py
test_ha_verify_fsm_recovering Skipped 0.02 test_hostha_simulator.py
test_ha_verify_fsm_fenced Skipped 0.02 test_hostha_simulator.py
test_ha_verify_fsm_degraded Skipped 0.03 test_hostha_simulator.py
test_ha_verify_fsm_available Skipped 0.02 test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownership Skipped 0.06 test_hostha_simulator.py
test_ha_list_providers Skipped 0.03 test_hostha_simulator.py
test_ha_enable_feature_invalid Skipped 0.02 test_hostha_simulator.py
test_ha_disable_feature_invalid Skipped 0.03 test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzones Skipped 0.02 test_hostha_simulator.py
test_configure_ha_provider_valid Skipped 0.02 test_hostha_simulator.py
test_configure_ha_provider_invalid Skipped 0.03 test_hostha_simulator.py
test_deploy_vgpu_enabled_vm Skipped 0.03 test_deploy_vgpu_enabled_vm.py
test_3d_gpu_support Skipped 0.04 test_deploy_vgpu_enabled_vm.py

@rohityadavcloud
Copy link
Member Author

@blueorangutan test

@blueorangutan
Copy link

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@borisstoyanov
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

Trillian test result (tid-1723)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55824 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2336-t1723-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_host_annotations.py
Intermitten failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermitten failure detected: /marvin/tests/smoke/test_iso.py
Intermitten failure detected: /marvin/tests/smoke/test_network.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermitten failure detected: /marvin/tests/smoke/test_templates.py
Intermitten failure detected: /marvin/tests/smoke/test_usage.py
Intermitten failure detected: /marvin/tests/smoke/test_volumes.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 59 look OK, 8 have error(s)

Test Result Time (s) Test File
test_01_vpc_site2site_vpn Failure 176.97 test_vpc_vpn.py
test_01_vpc_remote_access_vpn Failure 86.35 test_vpc_vpn.py
test_07_resize_fail Failure 15.51 test_volumes.py
test_04_extract_template Failure 5.14 test_templates.py
test_04_rvpc_privategw_static_routes Failure 289.12 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 168.32 test_privategw_acl.py
test_02_vpc_privategw_static_routes Failure 279.16 test_privategw_acl.py
test_01_vpc_privategw_acl Failure 56.90 test_privategw_acl.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
test_01_volume_usage Error 196.53 test_usage.py
test_05_create_template_with_no_checksum Error 65.74 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.67 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.91 test_templates.py
test_02_create_template_with_checksum_sha1 Error 65.77 test_templates.py
ContextSuite context=TestRedundantIsolateNetworks>:teardown Error 196.93 test_routers_network_ops.py
test_02_create_iso_with_checksum_sha256 Error 65.77 test_iso.py
test_05_add_annotation_for_invalid_entityType Error 0.08 test_host_annotations.py
test_change_service_offering_for_vm_with_snapshots Skipped 0.00 test_vm_snapshots.py
test_09_copy_delete_template Skipped 0.02 test_templates.py
test_06_copy_template Skipped 0.00 test_templates.py
test_static_role_account_acls Skipped 0.03 test_staticroles.py
test_11_ss_nfs_version_on_ssvm Skipped 0.02 test_ssvm.py
test_01_scale_vm Skipped 0.00 test_scale_vm.py
test_01_primary_storage_iscsi Skipped 0.14 test_primary_storage.py
test_vm_nic_adapter_vmxnet3 Skipped 0.00 test_nic_adapter_type.py
test_03_nic_multiple_vmware Skipped 1.25 test_nic.py
test_nested_virtualization_vmware Skipped 0.00 test_nested_virtualization.py
test_06_copy_iso Skipped 0.00 test_iso.py
test_list_ha_for_host_valid Skipped 0.07 test_hostha_simulator.py
test_list_ha_for_host_invalid Skipped 0.06 test_hostha_simulator.py
test_list_ha_for_host Skipped 0.04 test_hostha_simulator.py
test_hostha_enable_feature_without_setting_provider Skipped 0.06 test_hostha_simulator.py
test_hostha_enable_feature_valid Skipped 0.08 test_hostha_simulator.py
test_hostha_disable_feature_valid Skipped 0.06 test_hostha_simulator.py
test_hostha_configure_invalid_provider Skipped 0.06 test_hostha_simulator.py
test_hostha_configure_default_driver Skipped 0.06 test_hostha_simulator.py
test_ha_verify_fsm_recovering Skipped 0.06 test_hostha_simulator.py
test_ha_verify_fsm_fenced Skipped 0.07 test_hostha_simulator.py
test_ha_verify_fsm_degraded Skipped 0.06 test_hostha_simulator.py
test_ha_verify_fsm_available Skipped 0.06 test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownership Skipped 0.04 test_hostha_simulator.py
test_ha_list_providers Skipped 0.04 test_hostha_simulator.py
test_ha_enable_feature_invalid Skipped 0.05 test_hostha_simulator.py
test_ha_disable_feature_invalid Skipped 0.05 test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzones Skipped 0.06 test_hostha_simulator.py
test_configure_ha_provider_valid Skipped 0.06 test_hostha_simulator.py
test_configure_ha_provider_invalid Skipped 0.06 test_hostha_simulator.py
test_deploy_vgpu_enabled_vm Skipped 0.04 test_deploy_vgpu_enabled_vm.py
test_3d_gpu_support Skipped 0.06 test_deploy_vgpu_enabled_vm.py

@rohityadavcloud
Copy link
Member Author

Due to several failures, I'll close this one now and may work in future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marcaurele marcaurele marcaurele approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@rohityadavcloud @borisstoyanov @blueorangutan @marcaurele