Slow X start as it waits for autostart VMs

[3hhh]

Qubes OS version (e.g., R3.2):

3.2, 4.0rc1

Description

Somewhat since 2.x the boot process is significantly slowed down as all VMs flagged to autostart (at least sys-net & sys-firewall) are required to start before the X server is started.

This is inconvenient for the following reasons:

• Users may want to use a VM first which doesn't require networking.
• If an error occurs during the startup of one of the autostart VMs, the user waited in vain.
• Startup of further VMs the user might want to start during his current session is delayed further.

[marmarek]

In 3.2 this is required to avoid various races between GUI startup, X start and VM startup. In 4.0 theoretically it shouldn't be needed. But it still may be useful feature: when you log in, your system is fully operational. If you don't want to wait, disable autostart of those VMs. They will be started when required (application launch) anyway.

…

-- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[marmarek]

In 3.2 this is required to avoid various races between GUI startup, X start and VM startup. In 4.0 theoretically it shouldn't be needed. But it still may be useful feature: when you log in, your system is fully operational. If you don't want to wait, disable autostart of those VMs. They will be started when required (application launch) anyway.

…

-- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[3hhh]

In 3.2 this is required to avoid various races between GUI startup, X
start and VM startup. In 4.0 theoretically it shouldn't be needed. But
it still may be useful feature: when you log in, your system is fully
operational.

That's the only plus and it doesn't even hold if some error occurred (in 3.2 50% of the time my sys-net was not started, in 4.0rc1 in 20% of boots sys-firewall or sys-net fails to start).

I'd even say the login time (the 5s the user needs to type his password to pass the dm login) could e.g. be used to startup VMs.

If you don't want to wait, disable autostart of those VMs. They will be
started when required (application launch) anyway.

Yes, I know I can do it all for myself.

However from a bigger perspective I think that the default settings of an OS should have performance in mind as long as security is not compromised. Every user notices the time it takes for his OS to boot until X is started (whatever happens afterwards tends to be noticed less). If you can reduce this by 40-50%, that's something many people will see as an improvement.

[3hhh]

In 3.2 this is required to avoid various races between GUI startup, X
start and VM startup. In 4.0 theoretically it shouldn't be needed. But
it still may be useful feature: when you log in, your system is fully
operational.

That's the only plus and it doesn't even hold if some error occurred (in 3.2 50% of the time my sys-net was not started, in 4.0rc1 in 20% of boots sys-firewall or sys-net fails to start).

I'd even say the login time (the 5s the user needs to type his password to pass the dm login) could e.g. be used to startup VMs.

If you don't want to wait, disable autostart of those VMs. They will be
started when required (application launch) anyway.

Yes, I know I can do it all for myself.

However from a bigger perspective I think that the default settings of an OS should have performance in mind as long as security is not compromised. Every user notices the time it takes for his OS to boot until X is started (whatever happens afterwards tends to be noticed less). If you can reduce this by 40-50%, that's something many people will see as an improvement.

[3hhh]

Ok, so I did some tests to get more concrete numbers:

Methodology

• test system: Lenovo T530, i5 3rd Gen, 16 GB RAM, SSD, Qubes 4.0rc1
• 4 VMs in autostart: sys-usb, sys-net, sys-firewall, sys-vpn (proxy to sys-firewall)
• "X time" := time measured right after the luks password was typed in and until the X server showed up (in my case until awesome was started)
• "VM time" := time the aforementioned VMs required to start when the Qubes autostart feature was not used (I started a script from my awesome rc.lua which took the time & started the VMs instead)
• WM logon was set to autologin, i.e. no time was used to type in any password
• each test scenario was measured two times
• "chain loading" := sys-net, sys-firewall and sys-vpn were started one after another, sys-usb in parallel
• "parallel loading" := sys-net, sys-firewall, sys-vpn and sys-usb were all started in parallel

Measurements

Qubes Default Boot

X time: 43s, 44s

Manual autostart, chain loading

X time: 7s, 7s
VM time: 47s, 48s

Manual autostart, parallel loading

X time: 7s, 7s
VM time: 47s, 47s

Conclusions

• The Qubes default boot seems to be faster by ~10s for the test system if measured in total. Some small parts of this may be caused by my bad implementation (awesome & the VMs could be started in parallel, 4 times using qvm-start --skip-if-running seems to take ~2s if the VMs are running/nothing has to be done).
• Seeing X in 7s rather than 44s can be an improvement for some users (being able to start further VMs & use dom0).
• Having a fully functional system in 44s might be preferred by other users.
• Errors during VM startup will further increase the Qubes Default boot time (I think the default timeout is at 90s per VM or so, i.e. booting will then take 90s+).
• Boot time for both scenarios could be improved by starting the VMs during the WM password prompt (was disabled during the tests).
• Chain loading has no advantage over parallel loading even though it seems more clever at first sight to keep the dependencies in mind.

Unfortunately this still leaves the topic controversial, but should help make both positions more easy to understand.

[3hhh]

Ok, so I did some tests to get more concrete numbers:

Methodology

• test system: Lenovo T530, i5 3rd Gen, 16 GB RAM, SSD, Qubes 4.0rc1
• 4 VMs in autostart: sys-usb, sys-net, sys-firewall, sys-vpn (proxy to sys-firewall)
• "X time" := time measured right after the luks password was typed in and until the X server showed up (in my case until awesome was started)
• "VM time" := time the aforementioned VMs required to start when the Qubes autostart feature was not used (I started a script from my awesome rc.lua which took the time & started the VMs instead)
• WM logon was set to autologin, i.e. no time was used to type in any password
• each test scenario was measured two times
• "chain loading" := sys-net, sys-firewall and sys-vpn were started one after another, sys-usb in parallel
• "parallel loading" := sys-net, sys-firewall, sys-vpn and sys-usb were all started in parallel

Measurements

Qubes Default Boot

X time: 43s, 44s

Manual autostart, chain loading

X time: 7s, 7s
VM time: 47s, 48s

Manual autostart, parallel loading

X time: 7s, 7s
VM time: 47s, 47s

Conclusions

• The Qubes default boot seems to be faster by ~10s for the test system if measured in total. Some small parts of this may be caused by my bad implementation (awesome & the VMs could be started in parallel, 4 times using qvm-start --skip-if-running seems to take ~2s if the VMs are running/nothing has to be done).
• Seeing X in 7s rather than 44s can be an improvement for some users (being able to start further VMs & use dom0).
• Having a fully functional system in 44s might be preferred by other users.
• Errors during VM startup will further increase the Qubes Default boot time (I think the default timeout is at 90s per VM or so, i.e. booting will then take 90s+).
• Boot time for both scenarios could be improved by starting the VMs during the WM password prompt (was disabled during the tests).
• Chain loading has no advantage over parallel loading even though it seems more clever at first sight to keep the dependencies in mind.

Unfortunately this still leaves the topic controversial, but should help make both positions more easy to understand.

[3hhh]

Side note: VMs were running in hvm mode except for sys-usb. With pv mode everything is significantly faster.

[3hhh]

Side note: VMs were running in hvm mode except for sys-usb. With pv mode everything is significantly faster.

[3hhh]

I recently stumbled across an article in one of the biggest German IT magazines [1] in the context of an ORWL test [2] and it said:

Qubes OS boots very slowly; it takes 45s until you see the login screen.
(translated from German)

Pretty much my point. ;-)

[1] c't 20/2017, p. 64, https://shop.heise.de/katalog/ct-20-2017
[2] https://orwl.org/

[3hhh]

I recently stumbled across an article in one of the biggest German IT magazines [1] in the context of an ORWL test [2] and it said:

Qubes OS boots very slowly; it takes 45s until you see the login screen.
(translated from German)

Pretty much my point. ;-)

[1] c't 20/2017, p. 64, https://shop.heise.de/katalog/ct-20-2017
[2] https://orwl.org/

[marmarek]

There is one thing you can try: start VMs in parallel with X and password prompt. Create /etc/systemd/system/qubes-vm@.service.d/order.conf with:

[Unit]
Before=

In 3.2 this would break VM's GUI, but in 4.0 it should be safe(r).
This should give at least an impression of faster system startup.

[marmarek]

There is one thing you can try: start VMs in parallel with X and password prompt. Create /etc/systemd/system/qubes-vm@.service.d/order.conf with:

[Unit]
Before=

In 3.2 this would break VM's GUI, but in 4.0 it should be safe(r).
This should give at least an impression of faster system startup.

[rustybird]

There is one thing you can try: start VMs in parallel with X and password prompt. Create /etc/systemd/system/qubes-vm@.service.d/order.conf with:

[Unit]
Before=

I recently attempted this on R4.0 rc4, but systemd can't reset orderings via a drop-in. Only new ones can be added - see systemctl show --property=Before qubes-vm@whatever.service.

Anyway, patching qubes-vm@.service to remove Before=systemd-user-sessions.service caused the Network Manager tray icon for sys-net to be missing (until I manually started the GUI daemon) on most startups.

[rustybird]

There is one thing you can try: start VMs in parallel with X and password prompt. Create /etc/systemd/system/qubes-vm@.service.d/order.conf with:

[Unit]
Before=

I recently attempted this on R4.0 rc4, but systemd can't reset orderings via a drop-in. Only new ones can be added - see systemctl show --property=Before qubes-vm@whatever.service.

Anyway, patching qubes-vm@.service to remove Before=systemd-user-sessions.service caused the Network Manager tray icon for sys-net to be missing (until I manually started the GUI daemon) on most startups.

[donob4n]

Maybe an easy solution for 3.2 could be that autostarting VMs wait for X instead the opposite.

It will take more total time but the user experience will boost if you get the password prompt or your desktop (I use autologin since there is already a luks passphrase for booting) in ~7 seconds instead 40...

[donob4n]

Maybe an easy solution for 3.2 could be that autostarting VMs wait for X instead the opposite.

It will take more total time but the user experience will boost if you get the password prompt or your desktop (I use autologin since there is already a luks passphrase for booting) in ~7 seconds instead 40...

[3hhh]

Maybe an easy solution for 3.2 could be that autostarting VMs wait for X instead the opposite.

Yes, that's an option if you use autologin anyway.

Othe than that if one wants to use the password typing time, I see the following options for applications that require X:

1. in such VMs: Start applications using something such as wait until X available; [application start].

2. in dom0: Use the X desktop files to run qvm-run [VM] [application] as these are only executed after X startup anyway.

For nm-applet in particular there's also a cli available that doesn't need X. But then of course you're losing that handy tray app...

[3hhh]

Maybe an easy solution for 3.2 could be that autostarting VMs wait for X instead the opposite.

Yes, that's an option if you use autologin anyway.

Othe than that if one wants to use the password typing time, I see the following options for applications that require X:

1. in such VMs: Start applications using something such as wait until X available; [application start].

2. in dom0: Use the X desktop files to run qvm-run [VM] [application] as these are only executed after X startup anyway.

For nm-applet in particular there's also a cli available that doesn't need X. But then of course you're losing that handy tray app...

[3hhh]

@donob4n One of the reasons I don't like autologin atm: If your X server crashes with a screen lock on, it'll automatically restart and you may end up with an unlocked machine unless you have some clever screen locker checking for a crash during each X server startup...

[3hhh]

@donob4n One of the reasons I don't like autologin atm: If your X server crashes with a screen lock on, it'll automatically restart and you may end up with an unlocked machine unless you have some clever screen locker checking for a crash during each X server startup...

[marmarek]

Maybe an easy solution for 3.2 could be that autostarting VMs wait for X instead the opposite.

You can simply disable "autostart" option, then add own session startup command (in Xfce go to Settings Manager -> Session and Startup) like qvm-start some-vm or even qvm-run -a some-vm some-app.

[marmarek]

Maybe an easy solution for 3.2 could be that autostarting VMs wait for X instead the opposite.

You can simply disable "autostart" option, then add own session startup command (in Xfce go to Settings Manager -> Session and Startup) like qvm-start some-vm or even qvm-run -a some-vm some-app.

[donob4n]

@3hhh

One of the reasons I don't like autologin atm: If your X server crashes with a screen lock on, it'll automatically restart and you may end up with an unlocked machine unless you have some clever screen locker checking for a crash during each X server startup...

Good to know, thanks.

@marmarek

You can simply disable "autostart" option, then add own session startup command (in Xfce go to Settings Manager -> Session and Startup) like qvm-start some-vm or even qvm-run -a some-vm some-app.

After some play with systemd and got GUI problems, I finally used this way and it works fine.

I will test on 4.0. The boot experience is really better and if it doesn't have problems should be default behavior right?

[donob4n]

@3hhh

One of the reasons I don't like autologin atm: If your X server crashes with a screen lock on, it'll automatically restart and you may end up with an unlocked machine unless you have some clever screen locker checking for a crash during each X server startup...

Good to know, thanks.

@marmarek

You can simply disable "autostart" option, then add own session startup command (in Xfce go to Settings Manager -> Session and Startup) like qvm-start some-vm or even qvm-run -a some-vm some-app.

After some play with systemd and got GUI problems, I finally used this way and it works fine.

I will test on 4.0. The boot experience is really better and if it doesn't have problems should be default behavior right?

[3hhh]

Same request: https://groups.google.com/forum/#!topic/qubes-users/6QNAPDeMp5A

[3hhh]

Same request: https://groups.google.com/forum/#!topic/qubes-users/6QNAPDeMp5A