Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upUnsafe firewall fallback defaults #3269
Comments
andrewdavidwong
added
bug
C: core
labels
Nov 2, 2017
andrewdavidwong
added this to the Release 4.0 milestone
Nov 2, 2017
3hhh
referenced this issue
Nov 3, 2017
Closed
qubes-firewall may fail to start without downstream DNS #3277
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
3hhh
Nov 4, 2017
This may lead to VPN or Tor leaks unless the leaks are prevented further downstream or the user code does some leak prevention in the proxy VM itself.
@adrelanos fyi
3hhh
commented
Nov 4, 2017
|
This may lead to VPN or Tor leaks unless the leaks are prevented further downstream or the user code does some leak prevention in the proxy VM itself. @adrelanos fyi |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
Nov 17, 2017
Member
One may decide to not use qubes-firewall service at all - for example this is the case for Whonix Gateway. This means that not starting qubes-firewall should not lead to unusable network (in downstream VMs). But when started, indeed it should have safe fallback for cases like crash.
|
One may decide to not use qubes-firewall service at all - for example this is the case for Whonix Gateway. This means that not starting qubes-firewall should not lead to unusable network (in downstream VMs). But when started, indeed it should have safe fallback for cases like crash. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
3hhh
Nov 18, 2017
One may decide to not use qubes-firewall service at all - for example this is the case for Whonix Gateway. This means that not starting qubes-firewall should not lead to unusable network (in downstream VMs).
(I guess you meant upstream.)
No, the default should lead to unusable network as well from my point of view.
I'd recommend supporting a straightforward opt-in switch for e.g. qvm-prefs that basically enables passthrough for the default firewall settings. Or a special command for qubes-firewall to shut it down in a way that enables passthrough for everything.
Why? It's less prone to misconfigurations, easy to spot (no network) and fix. A few user questions will pop up here and there, but that'll be it. The other default option is more prone to misconfigurations and hard to spot.
Anyway the default firewall rules don't look like this behavior was intended; at least it's far from obvious.
But when started, indeed it should have safe fallback for cases like crash.
That for sure.
3hhh
commented
Nov 18, 2017
(I guess you meant upstream.) No, the default should lead to unusable network as well from my point of view. Anyway the default firewall rules don't look like this behavior was intended; at least it's far from obvious.
That for sure. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
unman
Nov 18, 2017
Member
(I guess you meant upstream.)
No, the VMs are downstream of the firewall - if it means anything in this context, you have the direction of travel reversed.
I dont think there's need for a special switch here, because I dont see what it adds. ProxyVM configured with default DENY on FORWARD chain, and a FAILUREACTION on firewall service configured to flush the FORWARD chain, (reinstate DENY policy), and then attempt service restart.
imo not starting qubes-firewall should lead to unusable network unless some other service is configured to provide firewall/proxy service, which means that the proxy defaults should be as above.
No, the VMs are downstream of the firewall - if it means anything in this context, you have the direction of travel reversed. I dont think there's need for a special switch here, because I dont see what it adds. ProxyVM configured with default DENY on FORWARD chain, and a FAILUREACTION on firewall service configured to flush the FORWARD chain, (reinstate DENY policy), and then attempt service restart. imo not starting qubes-firewall should lead to unusable network unless some other service is configured to provide firewall/proxy service, which means that the proxy defaults should be as above. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
Nov 19, 2017
Member
imo not starting qubes-firewall should lead to unusable network
qubes-firewall service is not data leak prevention mechanism. It is merely a guard against user errors. For network-related leaks prevention, use appropriate tool, like Whonix Gateway, or VPN scripts. Therefore the default (without qubes-firewall service running) should be usable network. If you want otherwise, there are mechanisms to achieve it. Simply adding appropriate rule in FORWARD chain, one time during boot, will do. qubes-firewall service in Qubes 4.0 will not touch your custom rules, it will only make changes in QBS-FORWARD chain (or qubes-firewall table in case of nftables). You can even remove/alter reference to it, to use it some other way.
But even if we'd like to have this behavior changed, it is much too late for 4.0. Such change have great potential to breaking various configurations (there are many: including different distributions, Whonix, VPN scripts etc). And debugging network-related problems, especially on Qubes (multiple VMs involved) is not a fun for many users.
What we can do now, as said earlier, is adjust qubes-firewall to block by default when the service is running (or crashed, but not intentionally stopped). And ensure it is started early enough - before enabling IP forwarding.
qubes-firewall service is not data leak prevention mechanism. It is merely a guard against user errors. For network-related leaks prevention, use appropriate tool, like Whonix Gateway, or VPN scripts. Therefore the default (without qubes-firewall service running) should be usable network. If you want otherwise, there are mechanisms to achieve it. Simply adding appropriate rule in But even if we'd like to have this behavior changed, it is much too late for 4.0. Such change have great potential to breaking various configurations (there are many: including different distributions, Whonix, VPN scripts etc). And debugging network-related problems, especially on Qubes (multiple VMs involved) is not a fun for many users. What we can do now, as said earlier, is adjust qubes-firewall to block by default when the service is running (or crashed, but not intentionally stopped). And ensure it is started early enough - before enabling IP forwarding. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
3hhh
Nov 19, 2017
No, the VMs are downstream of the firewall - if it means anything in this context, you have the direction of travel reversed.
Ok maybe that's something whonix specific (didn't use it so far).
For me upstream = direction of source VM, downstream = going to the Internet.
Just noticed that the root cause probably lies with the following forward chain rule
1 84 ACCEPT all -- vif+ * 0.0.0.0/0 0.0.0.0/0
This is somewhat irritating as the entire FORWARD chain looks as follows:
iptables -L -vn
[...]
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
452 266K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
415 28236 QBS-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- vif+ vif+ 0.0.0.0/0 0.0.0.0/0
1 84 ACCEPT all -- vif+ * 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
[...]
So there is a default drop rule on the FORWARD chain, but traffic doesn't ever hit it (I tested with a VPN VM on a debian-8 template, so no nftables interference there - just iptables). The first packet goes to the aforementioned ACCEPT rule and then it hits the RELATED,ESTABLISHED rule.
QBS-FORWARD is empty when the qubes-firewall is stopped and has a default drop rule in the end otherwise.
By the way: Stopping and starting the firewall can lead to then disallowed traffic going through anyway as on-going traffic hits the RELATED,ESTABLISHED allow rule then.
I guess there may also be some leaks during VM shutdown.
Thus I understand that you don't want to provide leak prevention, but I'd suggest to make it at least somewhat more easy to achieve wherever possible.
What we can do now, as said earlier, is adjust qubes-firewall to block by default when the service is running (or crashed, but not intentionally stopped). And ensure it is started early enough - before enabling IP forwarding.
I guess that would be hard: It'll mean that you'll have to try to differentiate between unintended crash and intended shutdown (is e.g. a process SIGTERM an intended shutdown? what about SIGKILL?) and then you'll have to implement the ACCEPT ALL rule only for the intended shutdown. It'll have to be there if the service was not started at all as well though. So yes, the only reasonable option is before IP forwarding.
3hhh
commented
Nov 19, 2017
•
Ok maybe that's something whonix specific (didn't use it so far). Just noticed that the root cause probably lies with the following forward chain rule This is somewhat irritating as the entire FORWARD chain looks as follows:
So there is a default drop rule on the FORWARD chain, but traffic doesn't ever hit it (I tested with a VPN VM on a debian-8 template, so no nftables interference there - just iptables). The first packet goes to the aforementioned ACCEPT rule and then it hits the RELATED,ESTABLISHED rule. QBS-FORWARD is empty when the qubes-firewall is stopped and has a default drop rule in the end otherwise. By the way: Stopping and starting the firewall can lead to then disallowed traffic going through anyway as on-going traffic hits the RELATED,ESTABLISHED allow rule then. I guess there may also be some leaks during VM shutdown. Thus I understand that you don't want to provide leak prevention, but I'd suggest to make it at least somewhat more easy to achieve wherever possible.
I guess that would be hard: It'll mean that you'll have to try to differentiate between unintended crash and intended shutdown (is e.g. a process SIGTERM an intended shutdown? what about SIGKILL?) and then you'll have to implement the ACCEPT ALL rule only for the intended shutdown. It'll have to be there if the service was not started at all as well though. So yes, the only reasonable option is before IP forwarding. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
Nov 19, 2017
Member
So there is a default drop rule on the FORWARD chain, but traffic doesn't ever hit it (I tested with a VPN VM on a debian-8 template, so no nftables interference there - just iptables). The first packet goes to the aforementioned ACCEPT rule and then it hits the RELATED,ESTABLISHED rule.
Yes, generally default firewall (without qubes-firewall involved) is to allow connections coming from connected VMs and deny new connections coming the other way.
Thus I understand that you don't want to provide leak prevention, but I'd suggest to make it at least somewhat more easy to achieve wherever possible.
See for example VPN documentation. All it takes is to insert your rules before default ones:
# Block forwarding of connections through upstream network device
# (in case the vpn tunnel breaks):
iptables -I FORWARD -o eth0 -j DROP
iptables -I FORWARD -i eth0 -j DROP
Yes, generally default firewall (without qubes-firewall involved) is to allow connections coming from connected VMs and deny new connections coming the other way.
See for example VPN documentation. All it takes is to insert your rules before default ones:
|
added a commit
to marmarek/qubes-core-agent-linux
that referenced
this issue
Nov 20, 2017
marmarek
closed this
in
marmarek/qubes-core-agent-linux@57a3c2d
Nov 20, 2017
added a commit
to marmarek/qubes-core-agent-linux
that referenced
this issue
Nov 20, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Nov 21, 2017
Automated announcement from builder-github
The package qubes-core-agent_4.0.13-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
qubesos-bot
commented
Nov 21, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
the
r4.0-jessie-cur-test
label
Nov 21, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Nov 21, 2017
Automated announcement from builder-github
The package core-agent-linux has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:
sudo yum update --enablerepo=qubes-vm-r4.0-current-testing
qubesos-bot
commented
Nov 21, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
the
r4.0-centos7-cur-test
label
Nov 21, 2017
qubesos-bot
referenced this issue
in QubesOS/updates-status
Nov 21, 2017
Closed
core-agent-linux v4.0.13 (r4.0) #308
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Nov 21, 2017
Automated announcement from builder-github
The package qubes-core-agent_4.0.13-1+deb9u1 has been pushed to the r4.0 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
qubesos-bot
commented
Nov 21, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
the
r4.0-stretch-cur-test
label
Nov 21, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Nov 21, 2017
Automated announcement from builder-github
The package python2-dnf-plugins-qubes-hooks-4.0.13-1.fc24 has been pushed to the r4.0 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:
sudo yum update --enablerepo=qubes-vm-r4.0-current-testing
qubesos-bot
commented
Nov 21, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
the
r4.0-fc24-cur-test
label
Nov 21, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Nov 21, 2017
Automated announcement from builder-github
The package python2-dnf-plugins-qubes-hooks-4.0.13-1.fc25 has been pushed to the r4.0 testing repository for the Fedora fc25 template.
To test this update, please install it with the following command:
sudo yum update --enablerepo=qubes-vm-r4.0-current-testing
qubesos-bot
commented
Nov 21, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
the
r4.0-fc25-cur-test
label
Nov 21, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Nov 21, 2017
Automated announcement from builder-github
The package python2-dnf-plugins-qubes-hooks-4.0.13-1.fc26 has been pushed to the r4.0 testing repository for the Fedora fc26 template.
To test this update, please install it with the following command:
sudo yum update --enablerepo=qubes-vm-r4.0-current-testing
qubesos-bot
commented
Nov 21, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
the
r4.0-fc26-cur-test
label
Nov 21, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
3hhh
Nov 25, 2017
Some bash code to fix the defaults for reference:
#installForwardDropDefaultRule
#will install a FORWARD default drop rule which is not installed by Qubes in proxy VMs by default (cf. qubes-issue #3269)
#as of 4.0rc2 the FORWARD chain in a debian-8 proxy VM looks as follows (WARNING: Fedora uses nftables and may be different!!!!):
#Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
# pkts bytes target prot opt in out source destination
#11781 9086K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
# 93 5196 QBS-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- vif+ vif+ 0.0.0.0/0 0.0.0.0/0
# 0 0 ACCEPT all -- vif+ * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
#This effectively allows all traffic to eth0 or tun0 by rule #4.
#Thus the below changes this situation to:
#Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
# pkts bytes target prot opt in out source destination
#11781 9086K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
# 93 5196 QBS-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 <--- NEW
# 0 0 DROP all -- vif+ vif+ 0.0.0.0/0 0.0.0.0/0 (might get removed, but is included in the above anyway)
# 0 0 ACCEPT all -- vif+ * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
function installForwardDropDefaultRule {
#this will produce:
# if no match is found --> an empty set of rules
# if the rule was already installed --> a set of rules where it is still installed & which doesn't have duplicates
# if the rules was not installed --> a set of rules where the rule is installed
#IMPORTANT: for any changes, test the aforementioned conditions with text files!!!
#:a --> label, N: include next line in pattern space, $!b except for the last line branch to label a --> read the whole text into pattern space
local insRules=""
insRules="$(iptables-save | sed -r -n ':a;N;$!ba;s/-A FORWARD -j QBS-FORWARD\n(-A FORWARD -j DROP|-A FORWARD -i vif\+ -o vif\+ -j DROP)/-A FORWARD -j QBS-FORWARD\n-A FORWARD -j DROP/p')"
[ $? -ne 0 ] || [ -z "$insRules" ] && errorOut "Failed to install the default drop iptables FORWARD rule! Maybe the Qubes rules changed?! Please investigate asap!"
echo "$insRules" | iptables-restore
#as of 4.0rc2 Qubes doesn't seem to touch the iptables rules anymore after VM start
}Just make sure to replace errorOut with your favorite error reporting function.
This doesn't just focus on eth0 as in the (imo mediocre) VPN doc, but also applies to e.g. tun0.
3hhh
commented
Nov 25, 2017
•
|
Some bash code to fix the defaults for reference: #installForwardDropDefaultRule
#will install a FORWARD default drop rule which is not installed by Qubes in proxy VMs by default (cf. qubes-issue #3269)
#as of 4.0rc2 the FORWARD chain in a debian-8 proxy VM looks as follows (WARNING: Fedora uses nftables and may be different!!!!):
#Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
# pkts bytes target prot opt in out source destination
#11781 9086K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
# 93 5196 QBS-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- vif+ vif+ 0.0.0.0/0 0.0.0.0/0
# 0 0 ACCEPT all -- vif+ * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
#This effectively allows all traffic to eth0 or tun0 by rule #4.
#Thus the below changes this situation to:
#Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
# pkts bytes target prot opt in out source destination
#11781 9086K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
# 93 5196 QBS-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 <--- NEW
# 0 0 DROP all -- vif+ vif+ 0.0.0.0/0 0.0.0.0/0 (might get removed, but is included in the above anyway)
# 0 0 ACCEPT all -- vif+ * 0.0.0.0/0 0.0.0.0/0
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
function installForwardDropDefaultRule {
#this will produce:
# if no match is found --> an empty set of rules
# if the rule was already installed --> a set of rules where it is still installed & which doesn't have duplicates
# if the rules was not installed --> a set of rules where the rule is installed
#IMPORTANT: for any changes, test the aforementioned conditions with text files!!!
#:a --> label, N: include next line in pattern space, $!b except for the last line branch to label a --> read the whole text into pattern space
local insRules=""
insRules="$(iptables-save | sed -r -n ':a;N;$!ba;s/-A FORWARD -j QBS-FORWARD\n(-A FORWARD -j DROP|-A FORWARD -i vif\+ -o vif\+ -j DROP)/-A FORWARD -j QBS-FORWARD\n-A FORWARD -j DROP/p')"
[ $? -ne 0 ] || [ -z "$insRules" ] && errorOut "Failed to install the default drop iptables FORWARD rule! Maybe the Qubes rules changed?! Please investigate asap!"
echo "$insRules" | iptables-restore
#as of 4.0rc2 Qubes doesn't seem to touch the iptables rules anymore after VM start
}Just make sure to replace This doesn't just focus on eth0 as in the (imo mediocre) VPN doc, but also applies to e.g. tun0. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Dec 11, 2017
Automated announcement from builder-github
The package qubes-core-agent_4.0.13-1+deb8u1 has been pushed to the r4.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
qubesos-bot
commented
Dec 11, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
r4.0-jessie-stable
and removed
r4.0-jessie-cur-test
labels
Dec 11, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Dec 11, 2017
Automated announcement from builder-github
The package qubes-core-agent_4.0.13-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian stretch template.
To install this update, please use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
qubesos-bot
commented
Dec 11, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
r4.0-stretch-stable
and removed
r4.0-stretch-cur-test
labels
Dec 11, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Dec 11, 2017
Automated announcement from builder-github
The package core-agent-linux has been pushed to the r4.0 stable repository for the Fedora centos7 template.
To install this update, please use the standard update command:
sudo yum update
qubesos-bot
commented
Dec 11, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
r4.0-centos7-stable
and removed
r4.0-centos7-cur-test
labels
Dec 11, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Dec 12, 2017
Automated announcement from builder-github
The package python2-dnf-plugins-qubes-hooks-4.0.13-1.fc24 has been pushed to the r4.0 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:
sudo yum update
qubesos-bot
commented
Dec 12, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
removed
the
r4.0-fc24-cur-test
label
Dec 12, 2017
qubesos-bot
added
the
r4.0-fc24-stable
label
Dec 12, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Dec 12, 2017
Automated announcement from builder-github
The package python2-dnf-plugins-qubes-hooks-4.0.13-1.fc25 has been pushed to the r4.0 stable repository for the Fedora fc25 template.
To install this update, please use the standard update command:
sudo yum update
qubesos-bot
commented
Dec 12, 2017
|
Automated announcement from builder-github The package
|
qubesos-bot
added
r4.0-fc25-stable
and removed
r4.0-fc25-cur-test
labels
Dec 12, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
qubesos-bot
Dec 12, 2017
Automated announcement from builder-github
The package python2-dnf-plugins-qubes-hooks-4.0.13-1.fc26 has been pushed to the r4.0 stable repository for the Fedora fc26 template.
To install this update, please use the standard update command:
sudo yum update
qubesos-bot
commented
Dec 12, 2017
|
Automated announcement from builder-github The package
|
3hhh commentedNov 1, 2017
Qubes OS version:
4.0rc2
Affected TemplateVMs:
Debian-8, i.e. I only tested without nftables support
Steps to reproduce the behavior:
systemctl stop qubes-firewallon sys-firewall or any proxy VMForgetting to start it, a crash or some other issue might also work.
Expected behavior:
Upstream VMs cannot access the Internet (safe fallback).
Actual behavior:
Upstream VMs can access whatever they want on the Internet (unsafe fallback).
General notes:
I had a look at the iptables rules to identify the one causing this behavior, but didn't notice any obvious one. I then tested with traffic and noticed that the counters for this rule increased:
-A INPUT -i lo -j ACCEPTMaybe it is applied due to the Masquerading?
Starting the qubes-firewall makes everything go back to normal.