Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot use interface managed by NetworkManager (NM) #374

Closed
luizluca opened this issue Aug 17, 2018 · 18 comments
Closed

cannot use interface managed by NetworkManager (NM) #374

luizluca opened this issue Aug 17, 2018 · 18 comments
Labels
medium Medium priority bug.

Comments

@luizluca
Copy link

luizluca commented Aug 17, 2018

Hello,

Since firewalld upgraded to 0.6.x, I cannot use NetworkManager interfaces with firewalld. I'm explicitily using iptables.

# grep ^Firewall /etc/firewalld/firewalld.conf
FirewallBackend=iptables

# nmcli connection
NAME                              UUID                                  TYPE      DEVICE 
xxxx            a2e01b3f-1af6-498a-97ed-926c949c623d  ethernet  eno1   

# nmcli connection show a2e01b3f-1af6-498a-97ed-926c949c623d | grep zone
connection.zone:                        public

# grep -i zone /etc/NetworkManager/system-connections/xxxx
zone=public

# grep -i zone /etc/sysconfig/network/ifcfg-eno1
ZONE=public

Everything looks ok, but:

# firewall-cmd --get-default-zone
public
# firewall-cmd --get-zone-of-interface=eno1
no zone
# firewall-cmd --get-active-zones
vms
  interfaces: vboxnet0 vboxnet1

# firewall-cmd --zone=public --add-interface=eno1
The interface is under control of NetworkManager and already bound to 'public'
The interface is under control of NetworkManager, setting zone to 'public'.
success

# iptables -L | grep Chain
Chain INPUT (policy ACCEPT)
Chain FORWARD (policy ACCEPT)
Chain OUTPUT (policy ACCEPT)
Chain FORWARD_IN_ZONES (1 references)
Chain FORWARD_IN_ZONES_SOURCE (1 references)
Chain FORWARD_OUT_ZONES (1 references)
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
Chain FORWARD_direct (1 references)
Chain FWDI_vms (2 references)
Chain FWDI_vms_allow (1 references)
Chain FWDI_vms_deny (1 references)
Chain FWDI_vms_log (1 references)
Chain FWDO_vms (2 references)
Chain FWDO_vms_allow (1 references)
Chain FWDO_vms_deny (1 references)
Chain FWDO_vms_log (1 references)
Chain INPUT_ZONES (1 references)
Chain INPUT_ZONES_SOURCE (1 references)
Chain INPUT_direct (1 references)
Chain IN_vms (2 references)
Chain IN_vms_allow (1 references)
Chain IN_vms_deny (1 references)
Chain IN_vms_log (1 references)
Chain OUTPUT_direct (1 references)

In the end, I have no iptables rules for public zone.

It works if I downgrade to firewalld-0.5.3:

# firewall-cmd --get-active-zones
public
  interfaces: eno1
vms
  interfaces: vboxnet0 vboxnet1

# firewall-cmd --get-zone-of-interface=eno1
public

# iptables -L | grep Chain
Chain INPUT (policy ACCEPT)
Chain FORWARD (policy ACCEPT)
Chain OUTPUT (policy ACCEPT)
Chain FORWARD_IN_ZONES (1 references)
Chain FORWARD_IN_ZONES_SOURCE (1 references)
Chain FORWARD_OUT_ZONES (1 references)
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
Chain FORWARD_direct (1 references)
Chain FWDI_public (2 references)
Chain FWDI_public_allow (1 references)
Chain FWDI_public_deny (1 references)
Chain FWDI_public_log (1 references)
Chain FWDI_vms (2 references)
Chain FWDI_vms_allow (1 references)
Chain FWDI_vms_deny (1 references)
Chain FWDI_vms_log (1 references)
Chain FWDO_public (2 references)
Chain FWDO_public_allow (1 references)
Chain FWDO_public_deny (1 references)
Chain FWDO_public_log (1 references)
Chain FWDO_vms (2 references)
Chain FWDO_vms_allow (1 references)
Chain FWDO_vms_deny (1 references)
Chain FWDO_vms_log (1 references)
Chain INPUT_ZONES (1 references)
Chain INPUT_ZONES_SOURCE (1 references)
Chain INPUT_direct (1 references)
Chain IN_public (2 references)
Chain IN_public_allow (1 references)
Chain IN_public_deny (1 references)
Chain IN_public_log (1 references)
Chain IN_vms (2 references)
Chain IN_vms_allow (1 references)
Chain IN_vms_deny (1 references)
Chain IN_vms_log (1 references)
Chain OUTPUT_direct (1 references)

I'm testing with these OpenSUSE Tumbleweed packages:

  • firewalld-lang-0.6.1-1.1.noarch
  • firewall-config-0.6.1-1.1.noarch
  • firewall-applet-0.6.1-1.1.noarch
  • python3-firewall-0.6.1-1.1.noarch
  • firewalld-0.6.1-1.1.noarch

Do I need to update NM to a specific version? Or change any settings to make it work? My NM connection settings explicitly set zone to public.

@erig0
Copy link
Collaborator

erig0 commented Aug 17, 2018

In the end, I have no iptables rules for public zone.

0.6.0 defaults to nftables unless you've changed FirewallBackend. Check the output of nft list ruleset.

@luizluca
Copy link
Author

@erig0 , I had it explicitly configured to use iptables:

# grep ^Firewall /etc/firewalld/firewalld.conf
FirewallBackend=iptables

I does configure iptables, but only for the zones that firewalld considers active (vms).
It's missing the rules related to zone public

@luizluca
Copy link
Author

Can I provide any more info? Keeping downgrading to 0.5.3 is not a very nice solution, specially when you use a rolling release (OpenSUSE Tumbleweed).

@hwoarang
Copy link
Contributor

Your output is confusing. On the top you provide information for eno1 but later on you query about eno0. Public zone is assigned to eno1 so when you query the zone for eno0 is not set which is correct. You mix eno1 and eno0 in various places

So in the end, what is the output of

firewall-cmd --zone=public --add-interface=eno1
firewall-cmd --get-zone-of-interface=eno1

What kind of rules do you expect to see in the public zone? Can you provide the iptables -L output?

@luizluca
Copy link
Author

@hdhoang , you are right. I mixed some interface names. That's why "firewall-cmd --get-zone-of-interface=eno0" was not returning a zone. I do not have eno0. I retested all commands and edited the original text. Still, the problem persists.

So, basic, the problem is that eno1, managed by NetworkManager, is not assigned to firewalld public zone and public zone rules are not added as there is no interface using it.

@luizluca
Copy link
Author

Some follow-up. I got this logs in journald after either firewalld or NetworkManager is restarted:

Aug 29 16:23:08 xxx firewalld[15305]: ERROR: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore v1.6.2: goto 'POST_public' is not a chain
                                                                      
                                                                      Error occurred at line: 22
                                                                      Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 29 16:23:08 xxx firewalld[15305]: ERROR: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain
                                                                      
                                                                      Error occurred at line: 29
                                                                      Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
Aug 29 16:23:08 xxx firewalld[15305]: ERROR: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain
                                                                      
                                                                      Error occurred at line: 29
                                                                      Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
Aug 29 16:23:08 xxx NetworkManager[14741]: <warn>  [1535570588.4639] firewall: [0x55f33ee0b770,change:"eno1"]: complete: request failed (COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain
                                                                           
                                                                           Error occurred at line: 29
                                                                           Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
                                                                           )

Some part of firewalld thinks POST_public chain exists. So, part of it know NetworkManager connections.

@luizluca
Copy link
Author

@erig0 @hwoarang ,

I can do the debug part if you give me some tips where to look. I can deal with python.

How firewalld gets interface zone from NM?

@erig0
Copy link
Collaborator

erig0 commented Sep 21, 2018

@luizluca, Thanks for reporting this. I hit this myself and after quite a bit of debugging found it to be an issue with firewalld's transaction mechanism. I'm in the process of finalizing a fix.

@erig0 erig0 closed this as completed in 2e53fab Sep 21, 2018
@luizluca
Copy link
Author

@erig0 , I'm sorry but it did not fix the problem.

I applied the patch on 0.6.2 and I still get nothing about default zone in iptables.

# firewall-cmd --version
0.6.2
# firewall-cmd --get-zone-of-interface=eno1
no zone
# iptables -L | grep ^'Chain'
Chain INPUT (policy ACCEPT)
Chain FORWARD (policy ACCEPT)
Chain OUTPUT (policy ACCEPT)
Chain FORWARD_IN_ZONES (1 references)
Chain FORWARD_IN_ZONES_SOURCE (1 references)
Chain FORWARD_OUT_ZONES (1 references)
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
Chain FORWARD_direct (1 references)
Chain FWDI_vms (2 references)
Chain FWDI_vms_allow (1 references)
Chain FWDI_vms_deny (1 references)
Chain FWDI_vms_log (1 references)
Chain FWDO_vms (2 references)
Chain FWDO_vms_allow (1 references)
Chain FWDO_vms_deny (1 references)
Chain FWDO_vms_log (1 references)
Chain INPUT_ZONES (1 references)
Chain INPUT_ZONES_SOURCE (1 references)
Chain INPUT_direct (1 references)
Chain IN_vms (2 references)
Chain IN_vms_allow (1 references)
Chain IN_vms_deny (1 references)
Chain IN_vms_log (1 references)
Chain OUTPUT_direct (1 references)
# grep -B1 -A12 "calling clear"  /usr/lib/python3.6/site-packages/firewall/core/fw_transaction.py
    def clear(self):
        # calling clear on a zone_transaction that was spawned from a
        # FirewallTransaction needs to clear the fw_transaction and all the
        # other zones otherwise we end up with a partially cleared transaction.
        if self.fw_transaction:
            super(FirewallTransaction, self.fw_transaction).clear()
            for zone in self.fw_transaction.zone_transactions.keys():
                super(FirewallZoneTransaction, self.fw_transaction.zone_transactions[zone]).clear()
                del self.fw_transaction.zone_transactions[zone].chains[:]
                del self.fw_transaction.zone_transactions[zone].modules[:]
        else:
            super(FirewallZoneTransaction, self).clear()
            del self.chains[:]
            del self.modules[:]

@luizluca
Copy link
Author

BTW, would it be possible to reopen this issue? I cannot do it.

erig0 added a commit that referenced this issue Sep 21, 2018
Just like FirewallZoneTransaction.execute() that was spawned from a
FirewallTransaction must call FirewallTransaction.exectue() we should
also make sure the same is done for clear(). Otherwise we can end up
with a partially cleared transaction. This gets really hairy if the
FirewallTransaction contains many instances of FirewallZoneTransaction
which is common during startup with non-default configuration.

Fixes: #374
(cherry picked from commit 2e53fab)
@luizluca
Copy link
Author

I did some more debug. The issue is not NetworkManager. There is two problems.

First, I never got the relevant error message. Running manually, I got this pretty traceback:

2018-09-21 14:51:49 DEBUG1: Setting policy to 'ACCEPT'
2018-09-21 14:51:49 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/firewall/server/decorators.py", line 53, in handle_exceptions
    return func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/firewall/server/firewalld.py", line 94, in start
    return self.fw.start()
  File "/usr/lib/python3.6/site-packages/firewall/core/fw.py", line 484, in start
    self._start()
  File "/usr/lib/python3.6/site-packages/firewall/core/fw.py", line 452, in _start
    use_transaction=transaction)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 470, in change_default_zone
    self.apply_zone_settings(new_zone, zone_transaction)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 348, in apply_zone_settings
    self.__zone_settings(True, zone, use_zone_transaction)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 328, in __zone_settings
    zone_transaction)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 676, in __rule
    zone_transaction)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 1652, in _rule_prepare
    enable, zone, proto, port, destination, rule)
  File "/usr/lib/python3.6/site-packages/firewall/core/ipXtables.py", line 926, in build_zone_source_ports_rules
    rule_fragment += self._rich_rule_source_fragment(rich_rule.source)
  File "/usr/lib/python3.6/site-packages/firewall/core/ipXtables.py", line 855, in _rich_rule_source_fragment
    flags = self._fw.zone.__ipset_match_flags(rich_source.ipset, "src")
AttributeError: 'FirewallZone' object has no attribute '_ip4tables__ipset_match_flags'

Further, I get the iptables error, because public zone was not created:

2018-09-21 14:56:24 DEBUG1: Setting zone of interface 'eno1' to 'public'
2018-09-21 14:56:24 ERROR: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore v1.6.2: goto 'POST_public' is not a chain

Error occurred at line: 22
Try `iptables-restore -h' or 'iptables-restore --help' for more information.

2018-09-21 14:56:24 ERROR: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain

Error occurred at line: 29
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.

2018-09-21 14:56:24 DEBUG1: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/firewall/server/decorators.py", line 68, in dbus_handle_exceptions
    return func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/firewall/server/firewalld.py", line 1167, in changeZoneOfInterface
    _zone = self.fw.zone.change_zone_of_interface(zone, interface, sender)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 457, in change_zone_of_interface
    _zone = self.add_interface(zone, interface, sender)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_zone.py", line 435, in add_interface
    zone_transaction.execute(True)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_transaction.py", line 267, in execute
    super(FirewallZoneTransaction, self).execute(enable)
  File "/usr/lib/python3.6/site-packages/firewall/core/fw_transaction.py", line 143, in execute
    raise FirewallError(errors.COMMAND_FAILED, errorMsg)
firewall.errors.FirewallError: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain

Error occurred at line: 29
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.


2018-09-21 14:56:24 ERROR: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain

Error occurred at line: 29
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.

The firewall systemd service file sends stdout and stderr to null. I don't know who must change (firewalld error output or systemd service file) but that kind of error (a backtrace) should definitely get into journalctl logs.

Now, back to the original problem. Poor NM is innocent here.

firewalld fails to use rich rules matching ipsets:

<zone>
  (...)
  <rule family="ipv4">
    <source ipset="ipsetA-v4"/>
    <service name="myservice"/>
    <log prefix="myservice:">
      <limit value="5/s"/>
    </log>
    <accept/>
  </rule>
  (...)
</zone>

<?xml version="1.0" encoding="utf-8"?>
<ipset type="hash:net">
  <short>ipsetA-v4</short>
  <description>xxx</description>
  <option name="family" value="inet"/>
  <entry>10.1.1.1</entry>
  <entry>10.1.1.2</entry>
</ipset>

I can reproduce it even with very simple rules and empty ipset:

# firewall-offline-cmd --new-ipset=ipset-A --type=hash:net
# firewall-offline-cmd --add-rich-rule 'rule family=ipv4 source ipset="ipset-A" accept'
# /usr/sbin/firewalld --nofork --nopid --debug

@erig0 erig0 reopened this Sep 21, 2018
@erig0
Copy link
Collaborator

erig0 commented Sep 21, 2018

Thanks for the reproducer. The rich rule error is the root cause. I will investigate.

@erig0 erig0 added the medium Medium priority bug. label Sep 21, 2018
@erig0
Copy link
Collaborator

erig0 commented Sep 21, 2018

This is due to python's special treatment of variables/functions with two leading underscores. Previously this function was only called within the same source file.

Can you try this patch?

diff --git a/src/firewall/core/fw_zone.py b/src/firewall/core/fw_zone.py
index 2d7943930e3e..ca90f7fba0d4 100644
--- a/src/firewall/core/fw_zone.py
+++ b/src/firewall/core/fw_zone.py
@@ -1519,7 +1519,7 @@ class FirewallZone(object):
     def __ipset_type(self, name):
         return self._fw.ipset.get_type(name)

-    def __ipset_match_flags(self, name, flag):
+    def _ipset_match_flags(self, name, flag):
         return ",".join([flag] * self._fw.ipset.get_dimension(name))

     def _check_ipset_applied(self, name):
diff --git a/src/firewall/core/ipXtables.py b/src/firewall/core/ipXtables.py
index 66af2a26eb2b..02a518d2938d 100644
--- a/src/firewall/core/ipXtables.py
+++ b/src/firewall/core/ipXtables.py
@@ -852,7 +852,7 @@ class ip4tables(object):
             rule_fragment += [ "-m", "set" ]
             if rich_source.invert:
                 rule_fragment.append("!")
-            flags = self._fw.zone.__ipset_match_flags(rich_source.ipset, "src")
+            flags = self._fw.zone._ipset_match_flags(rich_source.ipset, "src")
             rule_fragment += [ "--match-set", rich_source.ipset, flags ]

         return rule_fragment

@luizluca
Copy link
Author

luizluca commented Sep 21, 2018

I did some more digging. This dirt patch:

--- /usr/lib/python3.6/site-packages/firewall/core/fw_zone.py.old       2018-09-21 16:20:15.568692611 -0300
+++ /usr/lib/python3.6/site-packages/firewall/core/fw_zone.py   2018-09-21 16:25:18.600976200 -0300
@@ -1519,6 +1519,15 @@
     def __ipset_type(self, name):
         return self._fw.ipset.get_type(name)
 
+    def ipset_match_flags(self, name, flag):
+        return self.__ipset_match_flags(name, flag)
+
+    def _ipset_match_flags(self, name, flag):
+        return self.__ipset_match_flags(name, flag)
+
+    def _ip4tables__ipset_match_flags(self, name, flag):
+        return self.__ipset_match_flags(name, flag)
+
     def __ipset_match_flags(self, name, flag):
         return ",".join([flag] * self._fw.ipset.get_dimension(name))
 
--- /usr/lib/python3.6/site-packages/firewall/core/ipXtables.py.old     2018-09-21 16:22:13.296802867 -0300
+++ /usr/lib/python3.6/site-packages/firewall/core/ipXtables.py 2018-09-21 16:21:31.188763431 -0300
@@ -852,7 +852,13 @@
             rule_fragment += [ "-m", "set" ]
             if rich_source.invert:
                 rule_fragment.append("!")
-            flags = self._fw.zone.__ipset_match_flags(rich_source.ipset, "src")
+            try:
+                flags = self._fw.zone.ipset_match_flags(rich_source.ipset, "src")
+                flags = self._fw.zone._ipset_match_flags(rich_source.ipset, "src")
+                flags = self._fw.zone.__ipset_match_flags(rich_source.ipset, "src")
+            except Exception as e:
+                log.error(e)
+            exit(1)
             rule_fragment += [ "--match-set", rich_source.ipset, flags ]
 
         return rule_fragment

Shows that calling FirewallZone methods prefixed with two "_" will, in fact, prefix that call with an extra "_ip4tables". So, either renaming "__ipset_match_flags" to remove the two "_" prefix or creating a new "_ip4tables" prefixed one will workaround the problem.

I'm really really no python expert but this looks like some black magic metaprogramming going on here. Or, would it be a normal python 3.6 thing? Or even a python bug?

Update: escaped some '_' because of markdown syntax

@erig0
Copy link
Collaborator

erig0 commented Sep 21, 2018

I'm really really no python expert but this looks like some black magic metaprogramming going on here.

Pretty much. If it is prefixed with two underscores python will prepend the class name.
https://docs.python.org/3/tutorial/classes.html#private-variables

@luizluca
Copy link
Author

Can you try this patch?

It works!

Now, the only problem is the fact that backtrace are being ignored by default.

@erig0
Copy link
Collaborator

erig0 commented Sep 21, 2018

Now, the only problem is the fact that backtrace are being ignored by default.

Usually the exception is caught and a warning/error is logged. The backtrace is only logged if debug is enabled.

@erig0 erig0 closed this as completed in fa0bce3 Sep 21, 2018
@luizluca
Copy link
Author

Usually the exception is caught and a warning/error is logged. The backtrace is only logged if debug is enabled.

That didn't happen. I only saw ip6?tables-restore errors:

set 21 14:50:49 xxx systemd[1]: Starting firewalld - dynamic firewall daemon...
set 21 14:50:50 xxx systemd[1]: Started firewalld - dynamic firewall daemon.
set 21 14:50:51 xxx firewalld[24881]: ERROR: '/usr/sbin/iptables-restore -w -n' failed: iptables-restore v1.6.2: goto 'POST_public' is not a chain
                                                                      
                                                                      Error occurred at line: 30
                                                                      Try `iptables-restore -h' or 'iptables-restore --help' for more information.
set 21 14:50:51 xxx firewalld[24881]: ERROR: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not a chain
                                                                      
                                                                      Error occurred at line: 30
                                                                      Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
set 21 14:50:51 xxx firewalld[24881]: ERROR: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.6.2: goto 'POST_public' is not>
                                                                      
                                                                      Error occurred at line: 30
                                                                      Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
set 21 14:51:18 xxx systemd[1]: Stopping firewalld - dynamic firewall daemon...
set 21 14:51:22 xxx systemd[1]: Stopped firewalld - dynamic firewall daemon.

Even on debug, I only saw the backtrace at DEBUG1: and no ERROR: msg.

erig0 added a commit that referenced this issue Sep 21, 2018
Rename __ipset_match_flags() to _ipset_match_flags() so it may be used
outside the class. With the iptables backend this fixes rich rules that
match a source using an ipset.

Fixes: #374
(cherry picked from commit fa0bce3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
medium Medium priority bug.
Projects
None yet
Development

No branches or pull requests

3 participants