Template VMs often launch without networking support and need to be restarted

[nvesely]

This is a problem I am having specifically with my TemplateVMs, where sometimes I need to restart them to get connectivity. I unfortunately have not troubleshooted why this is yet. If you could give me some suggestions into how I might do that, I would certainly be willing.

[marmarek]

By default TemplateVMs are isolated from the network, the only available service is updates proxy. Not even DNS queries are allowed. If you have problems with updates proxy, check if the service is running (in sys-net aka NetVM):

systemctl status qubes-updates-proxy

If yes, try to restart the service (sudo systemctl restart ...). I've found that updates proxy loads DNS settings at its startup, so if you have no network access at that moment, it will not work. This bug is already fixed in upcoming update.

[marmarek]

By default TemplateVMs are isolated from the network, the only available service is updates proxy. Not even DNS queries are allowed. If you have problems with updates proxy, check if the service is running (in sys-net aka NetVM):

systemctl status qubes-updates-proxy

If yes, try to restart the service (sudo systemctl restart ...). I've found that updates proxy loads DNS settings at its startup, so if you have no network access at that moment, it will not work. This bug is already fixed in upcoming update.

[nvesely]

The interesting thing is that I am doing nothing in NetVM to fix this. The updates proxy is already running and sometimes my Debian template will update while I have to initially restart my Fedora template or vice-versa. I'll have launched both at roughly the same time.

Maybe the wrong place to ask this, but has Whonix-Gateway implemented it's own update proxy? I know the Whonix gateway and workstation templates don't use the regular update prxoy.

[nvesely]

The interesting thing is that I am doing nothing in NetVM to fix this. The updates proxy is already running and sometimes my Debian template will update while I have to initially restart my Fedora template or vice-versa. I'll have launched both at roughly the same time.

Maybe the wrong place to ask this, but has Whonix-Gateway implemented it's own update proxy? I know the Whonix gateway and workstation templates don't use the regular update prxoy.

[marmarek]

On Sat, Apr 25, 2015 at 03:00:35PM -0700, Noah Vesely wrote:

The interesting thing is that I am doing nothing in NetVM to fix this. The updates proxy is already running and sometimes my Debian template will update while I have to initially restart my Fedora template or vice-versa. I'll have launched both at roughly the same time.

Maybe some race condition in setting up routes in firewallvm? Check logs
there. Maybe this is already fixed by
marmarek/qubes-core-agent-linux@c49d928
?
Note that this fix (package qubes-core-vm-3.0.7) isn't available in
yum repo yet.

Maybe the wrong place to ask this, but has Whonix-Gateway implemented it's own update proxy? I know the Whonix gateway and workstation templates don't use the regular update prxoy.

Yes, whonix-gateway runs it's own updates proxy, which is directed to
the Tor. But regarding implementation it is exactly the same tinyproxy,
with identical configuration.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Sat, Apr 25, 2015 at 03:00:35PM -0700, Noah Vesely wrote:

The interesting thing is that I am doing nothing in NetVM to fix this. The updates proxy is already running and sometimes my Debian template will update while I have to initially restart my Fedora template or vice-versa. I'll have launched both at roughly the same time.

Maybe some race condition in setting up routes in firewallvm? Check logs
there. Maybe this is already fixed by
marmarek/qubes-core-agent-linux@c49d928
?
Note that this fix (package qubes-core-vm-3.0.7) isn't available in
yum repo yet.

Maybe the wrong place to ask this, but has Whonix-Gateway implemented it's own update proxy? I know the Whonix gateway and workstation templates don't use the regular update prxoy.

Yes, whonix-gateway runs it's own updates proxy, which is directed to
the Tor. But regarding implementation it is exactly the same tinyproxy,
with identical configuration.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[nvesely]

Where in particular would you suggest I look?

[nvesely]

Where in particular would you suggest I look?

[nrgaway]

On 25 April 2015 at 18:09, Marek Marczykowski-Górecki <
notifications@github.com> wrote:

On Sat, Apr 25, 2015 at 03:00:35PM -0700, Noah Vesely wrote:

The interesting thing is that I am doing nothing in NetVM to fix this.
The updates proxy is already running and sometimes my Debian template will
update while I have to initially restart my Fedora template or vice-versa.
I'll have launched both at roughly the same time.

Maybe some race condition in setting up routes in firewallvm? Check logs
there. Maybe this is already fixed by
marmarek/qubes-core-agent-linux@c49d928
?

There is an issue with qubes-core-vm-3.0.7
marmarek/qubes-core-agent-linux@c49d928

Note that this fix (package qubes-core-vm-3.0.7) isn't available in

yum repo yet.

Maybe the wrong place to ask this, but has Whonix-Gateway implemented
it's own update proxy? I know the Whonix gateway and workstation templates
don't use the regular update prxoy.

Yes, whonix-gateway runs it's own updates proxy, which is directed to
the Tor. But regarding implementation it is exactly the same tinyproxy,
with identical configuration.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

—
Reply to this email directly or view it on GitHub
#969 (comment)
.

[nrgaway]

On 25 April 2015 at 18:09, Marek Marczykowski-Górecki <
notifications@github.com> wrote:

On Sat, Apr 25, 2015 at 03:00:35PM -0700, Noah Vesely wrote:

The interesting thing is that I am doing nothing in NetVM to fix this.
The updates proxy is already running and sometimes my Debian template will
update while I have to initially restart my Fedora template or vice-versa.
I'll have launched both at roughly the same time.

Maybe some race condition in setting up routes in firewallvm? Check logs
there. Maybe this is already fixed by
marmarek/qubes-core-agent-linux@c49d928
?

There is an issue with qubes-core-vm-3.0.7
marmarek/qubes-core-agent-linux@c49d928

Note that this fix (package qubes-core-vm-3.0.7) isn't available in

yum repo yet.

Maybe the wrong place to ask this, but has Whonix-Gateway implemented
it's own update proxy? I know the Whonix gateway and workstation templates
don't use the regular update prxoy.

Yes, whonix-gateway runs it's own updates proxy, which is directed to
the Tor. But regarding implementation it is exactly the same tinyproxy,
with identical configuration.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

—
Reply to this email directly or view it on GitHub
#969 (comment)
.

[nvesely]

Sorry, I glanced over the notes above that commit that @marmarek had also linked earlier in the thread. I had a suspicion it was vif-router-qubes, but asked where to look because I wasn't seeing tinyproxy starting anywhere near where vif was crashing in journalctl and felt like I might be on the wrong track.

I just decided to reproduce the race condition and Debian failed again. Then I ran journalctl again and saw it was in fact vif. I was looking for keywords earlier instead of looking by time, which wasn't the right way to go about this. If I had read the script for the relevant systemd unit I would have realized how tinyproxy is not launched by some signal from dom0 saying "a templateVM had launched so get ready." I don't know why I considered it might work this way; the way it is actually implemented seems to make more sense (although I'm actually still figure out exactly what that way is--I need to dig into the Xen documentation and source a bit more when I get the time to figure out exactly how this thing is working).

Anyway, I'll wait for the update. I feel like implementing it myself might break something as I did a diff on master and r2 and realized that passing -w wasn't the only change.

[nvesely]

Sorry, I glanced over the notes above that commit that @marmarek had also linked earlier in the thread. I had a suspicion it was vif-router-qubes, but asked where to look because I wasn't seeing tinyproxy starting anywhere near where vif was crashing in journalctl and felt like I might be on the wrong track.

I just decided to reproduce the race condition and Debian failed again. Then I ran journalctl again and saw it was in fact vif. I was looking for keywords earlier instead of looking by time, which wasn't the right way to go about this. If I had read the script for the relevant systemd unit I would have realized how tinyproxy is not launched by some signal from dom0 saying "a templateVM had launched so get ready." I don't know why I considered it might work this way; the way it is actually implemented seems to make more sense (although I'm actually still figure out exactly what that way is--I need to dig into the Xen documentation and source a bit more when I get the time to figure out exactly how this thing is working).

Anyway, I'll wait for the update. I feel like implementing it myself might break something as I did a diff on master and r2 and realized that passing -w wasn't the only change.

[Zrubi]

I had similar issues before. The workaround was a simple netvm reassign to the affected template.
(Even if you not realy change the netvm reassign the same one from the CLI fixed the problem.)

[Zrubi]

I had similar issues before. The workaround was a simple netvm reassign to the affected template.
(Even if you not realy change the netvm reassign the same one from the CLI fixed the problem.)

[cfcs]

I have this problem with appvms as well.
From my vast collection of dom0 shellscripts fixing random Qubes quirks:

#!/usr/bin/env bash
for vm in $*; do
  qvm-prefs -s ${vm} netvm firewallvm &
done

[cfcs]

I have this problem with appvms as well.
From my vast collection of dom0 shellscripts fixing random Qubes quirks:

#!/usr/bin/env bash
for vm in $*; do
  qvm-prefs -s ${vm} netvm firewallvm &
done

[marmarek]

By default TemplateVMs are isolated from the network, the only available service is updates proxy. Not even DNS queries are allowed. If you have problems with updates proxy, check if the service is running (in sys-net aka NetVM):

systemctl status qubes-updates-proxy

If yes, try to restart the service (sudo systemctl restart ...). I've found that updates proxy loads DNS settings at its startup, so if you have no network access at that moment, it will not work. This bug is already fixed in upcoming update.

Regarding vif-route-qubes, check routing table in firewallvm - there should be a route to your template (get the IP from VM properties or read from the template terminal directly). If not - the problem is with vif-route-qubes

[marmarek]

By default TemplateVMs are isolated from the network, the only available service is updates proxy. Not even DNS queries are allowed. If you have problems with updates proxy, check if the service is running (in sys-net aka NetVM):

systemctl status qubes-updates-proxy

If yes, try to restart the service (sudo systemctl restart ...). I've found that updates proxy loads DNS settings at its startup, so if you have no network access at that moment, it will not work. This bug is already fixed in upcoming update.

Regarding vif-route-qubes, check routing table in firewallvm - there should be a route to your template (get the IP from VM properties or read from the template terminal directly). If not - the problem is with vif-route-qubes

[cfcs]

Isn't qubes-updates-proxy Fedora-specific?

By the way, while on the subject of networks -- right now I have a cron job that reloads my iptables config every minute since the qubes firewall seems to add rules automatically, breaking it. Is there a way to disable the firewall, or to make it only touch the QBS-prefixed chains instead of polluting the main INPUT OUTPUT FORWARD and DNAT chains?

[cfcs]

Isn't qubes-updates-proxy Fedora-specific?

By the way, while on the subject of networks -- right now I have a cron job that reloads my iptables config every minute since the qubes firewall seems to add rules automatically, breaking it. Is there a way to disable the firewall, or to make it only touch the QBS-prefixed chains instead of polluting the main INPUT OUTPUT FORWARD and DNAT chains?

[marmarek]

On Sun, May 03, 2015 at 10:05:15AM -0700, cfcs wrote:

Isn't qubes-updates-proxy Fedora-specific?

No, it is used for updates of every template (at least by default). This
is why it was renamed from qubes-yum-proxy.

By the way, while on the subject of networks -- right now I have a cron job that reloads my iptables config every minute since the qubes firewall seems to add rules automatically, breaking it. Is there a way to disable the firewall, or to make it only touch the QBS-prefixed chains instead of polluting the main INPUT OUTPUT FORWARD and DNAT chains?

You can simply add your changes in
/rw/config/qubes-firewall-user-script, which is called every time
qubes-firewall reloads iptables. Remember to make it executable.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Sun, May 03, 2015 at 10:05:15AM -0700, cfcs wrote:

Isn't qubes-updates-proxy Fedora-specific?

No, it is used for updates of every template (at least by default). This
is why it was renamed from qubes-yum-proxy.

By the way, while on the subject of networks -- right now I have a cron job that reloads my iptables config every minute since the qubes firewall seems to add rules automatically, breaking it. Is there a way to disable the firewall, or to make it only touch the QBS-prefixed chains instead of polluting the main INPUT OUTPUT FORWARD and DNAT chains?

You can simply add your changes in
/rw/config/qubes-firewall-user-script, which is called every time
qubes-firewall reloads iptables. Remember to make it executable.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[cfcs]

Thanks!

[cfcs]

Thanks!

[marmarek]

Since qubes-core-vm-3.0.8 is out for a few weeks, does the problem still exists?

[marmarek]

Since qubes-core-vm-3.0.8 is out for a few weeks, does the problem still exists?

[marmarek]

I assume the problem doesn't exist anymore. If happens again, please leave a comment.

[marmarek]

I assume the problem doesn't exist anymore. If happens again, please leave a comment.