Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firewalld state cause horrid performance during saltrun. #44979

Closed
hunkeelin opened this issue Dec 14, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@hunkeelin
Copy link

commented Dec 14, 2017

Some of my formula will have firewalld rules in it like the following:

dokuwiki_fw:
    firewalld.service:
        - name: dokuwiki
        - ports:
            - 8002/tcp

dokuwiki_fw_rule:
    firewalld.present:
        - name: public
        - prune_services: False
        - services:
            - dokuwiki

Some subset of my nodes will have serveral firewalld states to run during an highstate run. Upon inspecting debug I realize the firewalld module is consuming a lot of time. It is inefficient.

E.G

consider the following formula test

doku_default_pw:
    file.managed:
        - name: /tmp/test
        - source: salt://test/testfile

test_fw:
    firewalld.service:
        - name: test
        - ports:
            - 8002/tcp

dokuwiki_fw_rule:
    firewalld.present:
        - name: public
        - prune_services: False
        - services:
            - test

When I run salt-call state.apply test on the minion it takes an average of 10s for a no change occurrence.

local:

Summary for local
------------
Succeeded: 9
Failed:    0
------------
Total states run:     9
Total run time:  12.718 s

When I commented out the firewall lines. it takes significantly less time to run the state.

local:

Summary for local
------------
Succeeded: 7
Failed:    0
------------
Total states run:     7
Total run time:   1.577 s

@garethgreenaway

This comment has been minimized.

Copy link
Member

commented Dec 14, 2017

Thanks for the report. Sounds like the firewalld module(s) could benefit from the addition of mod_aggregate support.

@nhavens

This comment has been minimized.

Copy link
Contributor

commented May 9, 2018

@garethgreenaway, thanks for the mod_aggregate suggestion. I didn't know about that, but it could be really helpful in cases in which we need to run multiple firewall changes all at once.

Unfortunately, I don't think mod_aggregate will fix the slowness in the firewalld state module. Here's an example sls file with one invocation of the firewalld.present state function:

add firewall rule to test firewalld performance:
  firewalld.present:
    - name: public
    - ports:
      - 8080/tcp

Here is the output of a state.apply that made a change as well as 2 that did not (altered to hide username, hostname, and domain). Note that it took > 3 seconds to make a simple firewall rule change and > 2.3 seconds to confirm that the machine was already in the desired state:

user@master /srv/salt/test $ sudo salt 'testminion.example.org' state.apply test.fw
testminion.example.org:
----------
          ID: add firewall rule to test firewalld performance
    Function: firewalld.present
        Name: public
      Result: True
     Comment: 'public' was configured.
     Started: 12:24:29.134325
    Duration: 3217.489 ms
     Changes:   
              ----------
              ports:
                  ----------
                  new:
                      - 8080/tcp
                  old:

Summary for testminion.example.org
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:   3.217 s
user@master /srv/salt/test $ sudo salt 'testminion.example.org' state.apply test.fw
[sudo] password for user: 
testminion.example.org:
  Name: public - Function: firewalld.present - Result: Clean Started: - 12:44:31.981771 Duration: 2354.323 ms

Summary for testminion.example.org
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:   2.354 s
user@master /srv/salt/test $ sudo salt 'testminion.example.org' state.apply test.fw
testminion.example.org:
  Name: public - Function: firewalld.present - Result: Clean Started: - 12:44:37.940333 Duration: 2334.168 ms

Summary for testminion.example.org
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1 
Total run time:   2.334 s
user@master /srv/salt/test $ 

It looks like the firewalld.present state function makes many unnecessary calls to the firewalld execution module, which in turn runs the firewall-cmd command. Here the snip from the minion log file for the firewalld.present in the simple example above:

2018-05-09 13:13:52,076 [salt.state       :1795][INFO    ][30189] Executing state firewalld.present for [public]
2018-05-09 13:13:52,080 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --get-zones --permanent' in directory '/root'
2018-05-09 13:13:52,313 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --get-icmptypes --permanent' in directory '/root'
2018-05-09 13:13:52,532 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-icmp-blocks --permanent' in directory '/root'
2018-05-09 13:13:52,765 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-all --permanent' in directory '/root'
2018-05-09 13:13:52,997 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-ports --permanent' in directory '/root'
2018-05-09 13:13:53,242 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --add-port=8080/tcp --permanent' in directory '/root'
2018-05-09 13:13:53,478 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-forward-ports --permanent' in directory '/root'
2018-05-09 13:13:53,709 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-services --permanent' in directory '/root'
2018-05-09 13:13:53,939 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-interfaces --permanent' in directory '/root'
2018-05-09 13:13:54,184 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-sources --permanent' in directory '/root'
2018-05-09 13:13:54,416 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --zone=public --list-rich-rules --permanent' in directory '/root'
2018-05-09 13:13:54,658 [salt.loaded.int.module.cmdmod:385 ][INFO    ][30189] Executing command '/usr/bin/firewall-cmd --reload' in directory '/root'

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present icmp block handling
- Only call firewalld execution module icmp block functions if
necessary.
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present open port handling
- only call firewalld ports functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present port forward handling
- Only call firewalld port forward functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present service handling
- only call firewalld service functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present interface handling
- only call firewalld interface functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present source handling
- only call firewalld source functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present rich rule handling
- only call firewalld rich rule functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present open port handling
- only call firewalld ports functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present port forward handling
- Only call firewalld port forward functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present service handling
- only call firewalld service functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present interface handling
- only call firewalld interface functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present source handling
- only call firewalld source functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present rich rule handling
- only call firewalld rich rule functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present open port handling
- only call firewalld ports functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present port forward handling
- Only call firewalld port forward functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present service handling
- only call firewalld service functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present interface handling
- only call firewalld interface functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present source handling
- only call firewalld source functions if necessary
- Fixes saltstack#44979

nhavens added a commit to nhavens/salt that referenced this issue Sep 27, 2018

optimize firewalld.present rich rule handling
- only call firewalld rich rule functions if necessary
- Fixes saltstack#44979

@rallytime rallytime closed this in 8c88784 Oct 23, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.