Some VMs' clocks fall behind (separate problems with updates to qubes-rpc/policy definitions & qrexec-policy evaluation)

[jpouellet]

Qubes OS version:

R4, all latest pkgs from current-testing & security-testing as of time of reporting

Affected component(s):

At least a fedora-26 based vm, but problem seems to be in updating qrexec-policy for dom0

Steps to reproduce the behavior:

Start some VMs and put your computer to sleep for a while, notice their clocks become behind and fail to re-sync.

Expected behavior:

VMs keep accurate time.

Actual behavior:

VMs fall behind.

General notes:

One root cause / bug is partially that qubes-dom0-update does not update installed qrexec policies when they change in the source, but there is also another issue I have not found.

AFAICT clock syncing happens like this (at least for non-whonix VMs):

1. in each VM, qubes-sync-time service runs every 6 hours as long as clocksync service is not enabled, which is only enabled in sys-net (or maybe whatever your clockvm is set to?) (source)
2. qubes-sync-time service runs qvm-sync-clock as root (source)
3. qvm-sync-clock tries to run qubes.GetDate qrexec rpc service on '$default' (source)

Now things diverge. My /etc/qubes-rpc/policy/qubes.GetDate contained the following:

[user@dom0 ~]$ cat /etc/qubes-rpc/policy/qubes.GetDate
$tag:anon-vm $anyvm deny
## Note that policy parsing stops at the first match,
## so adding anything below "$anyvm $anyvm action" line will have no effect

## Please use a single # to start your custom comments

$anyvm	$anyvm	allow,target=sys-net

However, the current source contains the following:

## Note that policy parsing stops at the first match,
## so adding anything below "$anyvm $anyvm action" line will have no effect

## Please use a single # to start your custom comments

$anyvm	$anyvm	allow,target=dom0

Note difference of target=sys-net (old, & my config on disk even with all updates applied) vs target=dom0 (new).

This was changed in QubesOS/qubes-core-admin@bda9264 to add support for ClockVMs other than sys-net.

So, one bug is that some existing qubes installs still have target=sys-net instead of target=dom0, as mine did.

However, even after correcting that, for some reason (idk why), some VMs (not correlated with being tagged anon-vm, and still happens with that deny line commented out) get "Request refused" when attempting to run qubes.GetClock, but others do not.

For VMs in which qubes.GetClock fails, the clocks slowly get behind as the computer is asleep.

Related issues:

#1230 / QubesOS/qubes-core-agent-linux#47 / QubesOS/qubes-core-admin-linux#28 / QubesOS/qubes-core-admin#125

QubesOS/qubes-core-admin@bda9264

[jpouellet]

Okay, so the VMs with the delayed clocks had not been updated past the $defaultvm => @defaultvm change, so that was one half of the issue.

The general problem with how to update qrexec-policy on existing machines when it changes upstream still remains. I'll open a separate issue to more clearly track just that (without noise about clocks).

[jpouellet]

Okay, so the VMs with the delayed clocks had not been updated past the $defaultvm => @defaultvm change, so that was one half of the issue.

The general problem with how to update qrexec-policy on existing machines when it changes upstream still remains. I'll open a separate issue to more clearly track just that (without noise about clocks).