No Qubes tray icons when there are many VMs

[rustybird]

Qubes OS version:

R4.0 rc4 (latest current-testing)

Expected behavior:

The Qubes tray icons are shown.

Actual behavior:

They are missing, and the python3 -mqui.tray.* processes (which work if I run them manually in a terminal) defined in /etc/xdg/autostart are not running. After adding a wrapper to their .desktop files' Exec= values to capture all messages:

From qubes-domains.desktop:

ERROR:dbus.proxies:Introspect error on org.qubes.DomainManager1:/org/qubes/DomainManager1: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Traceback (most recent call last):
  File "/usr/lib64/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/qui/tray/domains.py", line 310, in <module>
    sys.exit(main())
  File "/usr/lib/python3.5/site-packages/qui/tray/domains.py", line 304, in main
    app = DomainTray('org.qubes.ui.tray.Domains')
  File "/usr/lib/python3.5/site-packages/qui/tray/domains.py", line 225, in __init__
    self.domain_manager = DomainManager()
  File "/usr/lib/python3.5/site-packages/qui/models/qubes.py", line 185, in __init__
    super().__init__(proxy, cls=Domain)
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 125, in __init__
    super(Properties, self).__init__(proxy, *args, **kwargs)
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 166, in __init__
    super(ObjectManager, self).__init__(proxy)
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 63, in __init__
    super(Model, self).__init__(self._init_interfaces())
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 74, in _init_interfaces
    xml_str = str(self.proxy.Introspect())
  File "/usr/lib64/python3.5/site-packages/dbus/proxies.py", line 70, in __call__
    return self._proxy_method(*args, **keywords)
  File "/usr/lib64/python3.5/site-packages/dbus/proxies.py", line 145, in __call__
    **keywords)
  File "/usr/lib64/python3.5/site-packages/dbus/connection.py", line 651, in call_blocking
    message, timeout)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
---- exit status 1 ----

From qubes-devices.desktop:

ERROR:dbus.proxies:Introspect error on org.qubes.DomainManager1:/org/qubes/DomainManager1: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Traceback (most recent call last):
  File "/usr/lib64/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/qui/tray/devices.py", line 22, in <module>
    DOMAINS = qui.models.qubes.DomainManager()
  File "/usr/lib/python3.5/site-packages/qui/models/qubes.py", line 185, in __init__
    super().__init__(proxy, cls=Domain)
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 125, in __init__
    super(Properties, self).__init__(proxy, *args, **kwargs)
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 166, in __init__
    super(ObjectManager, self).__init__(proxy)
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 63, in __init__
    super(Model, self).__init__(self._init_interfaces())
  File "/usr/lib/python3.5/site-packages/qui/models/dbus.py", line 74, in _init_interfaces
    xml_str = str(self.proxy.Introspect())
  File "/usr/lib64/python3.5/site-packages/dbus/proxies.py", line 70, in __call__
    return self._proxy_method(*args, **keywords)
  File "/usr/lib64/python3.5/site-packages/dbus/proxies.py", line 145, in __call__
    **keywords)
  File "/usr/lib64/python3.5/site-packages/dbus/connection.py", line 651, in call_blocking
    message, timeout)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
---- exit status 1 ----

[marmarek]

Do you see python3 -m qubesdbus.domain_manager process running, before trying to manually start the widget? If so, I guess that process is starting too slowly. You can try to kill it and check.
See also ~/.xsession-errors in dom0.

Does it happen during normal startup, or something like X server restart?

[marmarek]

Do you see python3 -m qubesdbus.domain_manager process running, before trying to manually start the widget? If so, I guess that process is starting too slowly. You can try to kill it and check.
See also ~/.xsession-errors in dom0.

Does it happen during normal startup, or something like X server restart?

[rustybird]

Do you see python3 -m qubesdbus.domain_manager process running, before trying to manually start the widget?

Yes.

If so, I guess that process is starting too slowly. You can try to kill it and check.

Thanks, that's it exactly.

It seems there are two problems:

• That there's any hard timeout at all for dom0 <-> dom0 communication, especially a short one
• The qubesdbus.domain_manager slowness (but to be fair, I somehow ended up with 110 VMs...)

See also ~/.xsession-errors in dom0.

Nothing interesting, except for the original error messages after I removed my debug wrapper script from qubes-*.desktop.

Does it happen during normal startup, or something like X server restart?

Normal startup.

[rustybird]

Do you see python3 -m qubesdbus.domain_manager process running, before trying to manually start the widget?

Yes.

If so, I guess that process is starting too slowly. You can try to kill it and check.

Thanks, that's it exactly.

It seems there are two problems:

• That there's any hard timeout at all for dom0 <-> dom0 communication, especially a short one
• The qubesdbus.domain_manager slowness (but to be fair, I somehow ended up with 110 VMs...)

See also ~/.xsession-errors in dom0.

Nothing interesting, except for the original error messages after I removed my debug wrapper script from qubes-*.desktop.

Does it happen during normal startup, or something like X server restart?

Normal startup.

[marmarek]

That there's any hard timeout at all for dom0 <-> dom0 communication, especially a short one

It's what dbus daemon does...

The qubesdbus.domain_manager slowness (but to be fair, I somehow ended up with 110 VMs...)

Hmm, I have ~90 VMs and that service starts in seconds for me... So, there must be something else causing its slow startup.

[marmarek]

That there's any hard timeout at all for dom0 <-> dom0 communication, especially a short one

It's what dbus daemon does...

The qubesdbus.domain_manager slowness (but to be fair, I somehow ended up with 110 VMs...)

Hmm, I have ~90 VMs and that service starts in seconds for me... So, there must be something else causing its slow startup.

[rustybird]

That there's any hard timeout at all for dom0 <-> dom0 communication, especially a short one

It's what dbus daemon does...

Which makes sense for a normal multi-user system. For Qubes dom0, not so much. Maybe the timeout could be disabled or bumped way up?

Hmm, I have ~90 VMs and that service starts in seconds for me... So, there must be something else causing its slow startup.

This is on an x230 (slowest 2.6 GHz variety) - you might have faster hardware?

domain_manager spends all its time here: ~50 seconds cold, ~40 seconds warm. In each iteration, it's the qubesdbus.serialize.domain_data() call in _proxify_domain() that takes 0.35 seconds per VM on average.

[rustybird]

That there's any hard timeout at all for dom0 <-> dom0 communication, especially a short one

It's what dbus daemon does...

Which makes sense for a normal multi-user system. For Qubes dom0, not so much. Maybe the timeout could be disabled or bumped way up?

Hmm, I have ~90 VMs and that service starts in seconds for me... So, there must be something else causing its slow startup.

This is on an x230 (slowest 2.6 GHz variety) - you might have faster hardware?

domain_manager spends all its time here: ~50 seconds cold, ~40 seconds warm. In each iteration, it's the qubesdbus.serialize.domain_data() call in _proxify_domain() that takes 0.35 seconds per VM on average.

[qubesos-bot]

Automated announcement from builder-github

The package qubes-desktop-linux-manager-4.0.7-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-desktop-linux-manager-4.0.7-1.fc25 has been pushed to the r4.0 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-desktop-linux-manager-4.0.7-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-desktop-linux-manager-4.0.7-1.fc25 has been pushed to the r4.0 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update