Disabling managing firewall - cloudstack-setup-management #4239

davidjumani · 2020-08-03T09:07:27Z

Description

Disabling managing iptables on the management server since the host might be using unsupported firewall management tools

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

andrijapanicsb · 2020-08-03T18:37:36Z

@davidjumani - thx for the PR. I see only comment added, but not really removing the actual code which injects raw iptable rules (iptables -A ....) and which runs the "iptables-save > /etc/sysconfig/iptables" command, and restarts iptables service...

I believe we want ZERO management of the iptables, as the rules from the /etc/sysconfig/iptables are overridden by the rules managed by firewalld (which is installed by default on CentOS 7/8)

/cc @PaulAngus @rhtyd @GabrielBrascher @wido @weizhouapache

wido · 2020-08-04T07:05:57Z

We do not want to manage iptables indeed. We (CloudStack) should never touch firewalls. That's up to the admin.

davidjumani · 2020-08-04T07:32:45Z

Thanks @andrijapanicsb @wido Made the changes!

davidjumani · 2020-08-04T07:33:45Z

@blueorangutan package

blueorangutan · 2020-08-04T07:34:57Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-04T08:06:25Z

Packaging result: ✔centos7 ✔debian. JID-1640

rohityadavcloud · 2020-08-10T07:45:19Z

LGTM
@blueorangutan package

blueorangutan · 2020-08-10T07:45:57Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

rohityadavcloud · 2020-08-10T11:11:30Z

this needs a test run as changes are in cloudstack-setup-management cc @davidjumani pl kick test when lab is online

DaanHoogland · 2020-08-11T07:21:36Z

@blueorangutan package

blueorangutan · 2020-08-11T07:22:56Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-11T08:28:52Z

Packaging result: ✔centos7 ✔debian. JID-1707

davidjumani · 2020-08-11T09:09:29Z

@blueorangutan test

blueorangutan · 2020-08-11T09:09:58Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

rohityadavcloud · 2020-08-11T11:33:22Z

@davidjumani env fails to deploy

rohityadavcloud · 2020-08-11T11:34:54Z

python/lib/cloudutils/syscfg.py

 class sysConfigServerRedhat(sysConfigServer):
    def __init__(self, glbEnv):
        super(sysConfigServerRedhat, self).__init__(glbEnv)
        self.svo = serviceOpsRedhat()
-        self.services = [firewallConfigServer(self)]


@davidjumani I think we should revert the previous behaviour and simply print the statement. Otherwise, env will not deploy. Previously the cloudstack-setup-management would open firewall, now that you've changed it - either (a) document this and fix trillian to open the ports both CentOS/Ubuntu envs, or (b) just print the line for information sake.

@rhtyd I was able to deploy a kvm env for basic testing, and everything seems to work. Will look into why it failed this time

okay @davidjumani when you confirm we'll be able to merge this

It's good to go. Env came up and smoke tests ran successfully!

davidjumani · 2020-08-11T15:58:27Z

@blueorangutan test

blueorangutan · 2020-08-11T15:58:57Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-08-12T03:31:57Z

Trillian test result (tid-2370)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39612 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4239-t2370-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_delete_kubernetes_supported_version	`Error`	1810.31	test_kubernetes_supported_versions.py

davidjumani · 2020-08-17T13:17:44Z

@blueorangutan package

blueorangutan · 2020-08-17T13:18:57Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-17T13:56:28Z

Packaging result: ✔centos7 ✔debian. JID-1772

rohityadavcloud · 2020-08-19T09:23:02Z

Just a reminder - pl kick test on centos8 and Ubuntu for this @davidjumani (use Jenkins if BO is limiting, thnx)

davidjumani · 2020-08-20T04:19:44Z

@rhtyd Jenkins was failing. Kicking them off now!

blueorangutan · 2020-08-28T04:23:45Z

Trillian test result (tid-2583)
Environment: kvm-ubuntu18 (x2), Advanced Networking with Mgmt server u18
Total time taken: 52615 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4239-t2583-kvm-ubuntu18.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_deploy_kubernetes_cluster	`Error`	0.10	test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster	`Error`	0.04	test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster	`Error`	0.04	test_kubernetes_clusters.py
test_11_migrate_vm	`Error`	48.10	test_vm_life_cycle.py
test_14_secure_to_secure_vm_migration	`Error`	97.56	test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	433.68	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	`Error`	3891.22	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	178.89	test_hostha_kvm.py

davidjumani · 2020-08-28T05:39:19Z

@blueorangutan test

blueorangutan · 2020-08-28T05:39:49Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-08-28T21:53:23Z

Trillian test result (tid-2584)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56315 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4239-t2584-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_deploy_kubernetes_cluster	`Error`	0.10	test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster	`Error`	0.07	test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster	`Error`	0.08	test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster	`Error`	0.09	test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster	`Error`	0.07	test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_oobm_issue_power_cycle	`Error`	17.36	test_outofbandmanagement.py
test_oobm_issue_power_on	`Error`	18.22	test_outofbandmanagement.py
test_oobm_issue_power_reset	`Error`	20.10	test_outofbandmanagement.py
test_oobm_issue_power_soft	`Error`	19.77	test_outofbandmanagement.py
test_oobm_issue_power_status	`Error`	21.67	test_outofbandmanagement.py
test_oobm_zchange_password	`Error`	8.93	test_outofbandmanagement.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	458.72	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Error`	347.90	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	`Error`	3971.48	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	174.70	test_hostha_kvm.py

blueorangutan · 2020-08-28T23:19:40Z

Trillian test result (tid-2591)
Environment: kvm-ubuntu18 (x2), Advanced Networking with Mgmt server u18
Total time taken: 49331 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4239-t2591-kvm-ubuntu18.zip
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 83 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_11_migrate_vm	`Error`	49.00	test_vm_life_cycle.py
test_14_secure_to_secure_vm_migration	`Error`	101.44	test_vm_life_cycle.py
test_hostha_kvm_host_fencing	`Error`	103.36	test_hostha_kvm.py

rohityadavcloud · 2020-09-01T08:23:38Z

@davidjumani can you check the centos7 failures, are those caused by regression or env related?

blueorangutan · 2020-09-02T06:57:49Z

Trillian test result (tid-2637)
Environment: kvm-ubuntu18 (x2), Advanced Networking with Mgmt server u18
Total time taken: 67832 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4239-t2637-kvm-ubuntu18.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 79 look OK, 6 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_04_extract_Iso	`Failure`	1082.38	test_iso.py
test_01_add_delete_kubernetes_supported_version	`Error`	1803.13	test_kubernetes_supported_versions.py
test_isolate_network_password_server	`Failure`	17.53	test_password_server.py
test_11_migrate_vm	`Error`	52.61	test_vm_life_cycle.py
test_14_secure_to_secure_vm_migration	`Error`	116.38	test_vm_life_cycle.py
test_02_redundant_VPC_default_routes	`Failure`	443.33	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Error`	385.90	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	`Error`	3893.27	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	224.88	test_hostha_kvm.py

davidjumani · 2020-09-02T11:46:19Z

@rhtyd Looking at the logs, the failures are env related issues

DaanHoogland · 2020-09-18T14:25:13Z

@blueorangutan package

blueorangutan · 2020-09-18T14:26:31Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-09-18T15:23:51Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2042

DaanHoogland · 2020-09-21T09:42:47Z

@blueorangutan test

blueorangutan · 2020-09-21T09:43:06Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

DaanHoogland

changes look good, I would like to see another smoke test result set

DaanHoogland · 2020-09-21T12:43:50Z

@blueorangutan test

blueorangutan · 2020-09-21T12:45:06Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-09-22T07:40:38Z

Trillian test result (tid-2826)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 65712 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4239-t2826-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_delete_kubernetes_supported_version	`Error`	909.24	test_kubernetes_supported_versions.py
test_02_vpc_privategw_static_routes	`Failure`	238.99	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	257.43	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	403.79	test_privategw_acl.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	422.75	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	172.66	test_hostha_kvm.py

davidjumani force-pushed the add-port-open-msg branch from 8f0c7b5 to 21ef12b Compare August 3, 2020 09:27

andrijapanicsb changed the title ~~Adding message to ensure ports are open~~ Adding message to ensure ports are open - cloudstack-setup-management Aug 3, 2020

davidjumani changed the title ~~Adding message to ensure ports are open - cloudstack-setup-management~~ Disabling managing firewall - cloudstack-setup-management Aug 4, 2020

rohityadavcloud added this to the 4.15.0.0 milestone Aug 5, 2020

rohityadavcloud closed this Aug 10, 2020

rohityadavcloud reopened this Aug 10, 2020

rohityadavcloud requested review from andrijapanicsb and wido August 10, 2020 07:45

wido approved these changes Aug 10, 2020

View reviewed changes

rohityadavcloud approved these changes Aug 10, 2020

View reviewed changes

rohityadavcloud reviewed Aug 11, 2020

View reviewed changes

davidjumani marked this pull request as ready for review August 12, 2020 04:43

davidjumani added 3 commits August 17, 2020 18:46

Adding message to ensure ports are open

0dff92b

Removing configuring iptables

f7447ea

Fixing merge conflict

94626f7

davidjumani force-pushed the add-port-open-msg branch from cba6231 to 94626f7 Compare August 17, 2020 13:17

svenvogel added component:management-server type:enhancement labels Aug 20, 2020

DaanHoogland approved these changes Sep 21, 2020

View reviewed changes

rohityadavcloud approved these changes Sep 22, 2020

View reviewed changes

rohityadavcloud merged commit ead9a34 into apache:master Sep 22, 2020

Disabling managing firewall - cloudstack-setup-management #4239

Disabling managing firewall - cloudstack-setup-management #4239

Conversation

davidjumani commented Aug 3, 2020 • edited Loading

Description

Types of changes

andrijapanicsb commented Aug 3, 2020

wido commented Aug 4, 2020

davidjumani commented Aug 4, 2020

davidjumani commented Aug 4, 2020

blueorangutan commented Aug 4, 2020

blueorangutan commented Aug 4, 2020

rohityadavcloud commented Aug 10, 2020

blueorangutan commented Aug 10, 2020

rohityadavcloud commented Aug 10, 2020

DaanHoogland commented Aug 11, 2020

blueorangutan commented Aug 11, 2020

blueorangutan commented Aug 11, 2020

davidjumani commented Aug 11, 2020

blueorangutan commented Aug 11, 2020

rohityadavcloud commented Aug 11, 2020

rohityadavcloud Aug 11, 2020

Choose a reason for hiding this comment

davidjumani Aug 11, 2020

Choose a reason for hiding this comment

rohityadavcloud Aug 12, 2020

Choose a reason for hiding this comment

davidjumani Aug 12, 2020

Choose a reason for hiding this comment

davidjumani commented Aug 11, 2020

blueorangutan commented Aug 11, 2020

blueorangutan commented Aug 12, 2020

davidjumani commented Aug 17, 2020

blueorangutan commented Aug 17, 2020

blueorangutan commented Aug 17, 2020

rohityadavcloud commented Aug 19, 2020

davidjumani commented Aug 20, 2020

blueorangutan commented Aug 28, 2020

davidjumani commented Aug 28, 2020

blueorangutan commented Aug 28, 2020

blueorangutan commented Aug 28, 2020

blueorangutan commented Aug 28, 2020

rohityadavcloud commented Sep 1, 2020

blueorangutan commented Sep 2, 2020

davidjumani commented Sep 2, 2020

DaanHoogland commented Sep 18, 2020

blueorangutan commented Sep 18, 2020

blueorangutan commented Sep 18, 2020

DaanHoogland commented Sep 21, 2020

blueorangutan commented Sep 21, 2020

DaanHoogland left a comment

Choose a reason for hiding this comment

DaanHoogland commented Sep 21, 2020

blueorangutan commented Sep 21, 2020

blueorangutan commented Sep 22, 2020

davidjumani commented Aug 3, 2020 •

edited

Loading