Run qemu-guest-agent to keep time in sync with host?

[jandubois]

There has been a question about containers falling behind in time when the host has been suspended: rancher-sandbox/rancher-desktop#836

I believe this would be fixed by installing the qemu-guest-agent, but haven't tested it (because I'm not sure how to set it up, but I've only looked at Alpine so far).

[totallyGreg]

I would like to be be able to do this so shutdowns are communicated properly and various stats are provided. I've had success installing the qemu-guest-agent in a k3os image running on proxmox, by adding this to the cloud-config file.

write_files:
- path: /etc/conf.d/qemu-guest-agent
  content: |-
    # Specifies the transport method used to communicate to QEMU on the host side
    # Default: virtio-serial
    #GA_METHOD="virtio-serial"
    # Specifies the device path for the communications back to QEMU on the host
    # Default: /dev/virtio-ports/org.qemu.guest_agent.0
    #GA_PATH="/dev/virtio-ports/org.qemu.guest_agent.0"
      GA_PATH=/dev/vport2p1
  owner: root
  permissions: '0644'

I am not sure how lima is communicating with qemu but if the device port path is known I can try it out.

[ko185061]

Im putting a little time towards trying some things with QEMU, I will update the thread if I figure anything out!

[ko185061]

It would appear that the issue is timedatectl cant actually hit the ntp server to sync against

[dmccaffery]

@AkihiroSuda - I looked into this and noticed a few things:

1. The qemu-guest-agent socket (virtio-serial) is not configured -- this is easy to resolve
2. There is no detection of guest pause/resume from the host

I'm happy to implement support for setting the clock but am not sure what the best way to go about it is within the design of this project. Should we implement settimeofday in the lima guest agent with a signal from the host agent? -- this removes the need for yet-another-socket (qga) to be set -- but it does require reinventing the wheel re: setting the time and the hwclock.

Alternatively, we can enable the qemu-guest-agent:

diff --git a/pkg/qemu/qemu.go b/pkg/qemu/qemu.go
index 71d633b..03a9c7f 100644
--- a/pkg/qemu/qemu.go
+++ b/pkg/qemu/qemu.go
@@ -345,6 +345,17 @@ func Cmdline(cfg Config) (string, []string, error) {
        args = append(args, "-chardev", fmt.Sprintf("socket,id=%s,path=%s,server=on,wait=off,logfile=%s", serialChardev, serialSock, serialLog))
        args = append(args, "-serial", "chardev:"+serialChardev)
 
+       // QEMU Guest Agent
+       qgaSock := filepath.Join(cfg.InstanceDir, filenames.QemuGuestAgentSock)
+       if err := os.RemoveAll(qgaSock); err != nil {
+               return "", nil, err
+       }
+
+       const qgaChardev = "char-qga"
+       args = append(args, "-chardev", fmt.Sprintf("socket,id=%s,path=%s,server=on,wait=off", qgaChardev, qgaSock))
+       args = append(args, "-device", "virtio-serial")
+       args = append(args, "-device", fmt.Sprintf("virtserialport,chardev=%s,name=org.qemu.guest_agent.0", qgaChardev))
+
        // We also want to enable vsock and virtfs here, but QEMU does not support vsock and virtfs for macOS hosts
 
        // QMP
diff --git a/pkg/store/filenames/filenames.go b/pkg/store/filenames/filenames.go
index 358b6ac..ca26b5f 100644
--- a/pkg/store/filenames/filenames.go
+++ b/pkg/store/filenames/filenames.go
@@ -33,6 +33,7 @@ const (
        SerialSock         = "serial.sock"
        SSHSock            = "ssh.sock"
        GuestAgentSock     = "ga.sock"
+       QemuGuestAgentSock = "qga.sock"
        HostAgentPID       = "ha.pid"
        HostAgentSock      = "ha.sock"
        HostAgentStdoutLog = "ha.stdout.log"

This enables us to use the existing guest-set-time command, but we still need some mechanism to detect resumes and invoke said command from the host agent.

I tried using timesyncd, but if the drift is too long, the clock is (seemingly) never corrected.

Happy to go down either path -- just looking for some guidance on your preference for implementation. Thank you!

[AkihiroSuda]

Should we implement settimeofday in the lima guest agent with a signal from the host agent?

SGTM, thanks

[chrisdostert]

Looks like this got closed with #490

[jandubois]

Looks like this got closed with #490

Yes, it did, and #490 is included in the v0.8.1 release. Please report back if it doesn't fix the issue for you!

[artem-karpenko-bolt]

Hello @jandubois,
I still experience the issue of clock skew between host and VM. In my case it is a 1-1.5 second difference which as I understand is within the delta being tracked in the relevant PR. In my case however this is crucial - my code uses local (host) time and compares it against time returned by VM (through MySQL's NOW() function). Having delta of 1 second (i.e. re-syncing clocks if time is different in 1+ second) would solve it for me and I guess for all cases where unix timestamp is used (i.e. second precision). What do you think?

On a second thought, current mechanism, even if we set delta to 1 sec, still has precision of 1 sec. Would it make sense to modify the logic to use millisecond precision when syncing time?

[spacedub]

Seems like it is not working for me: #1307
Apologies for opening a new ticket as I did not notice this one first - let me know if you prefer the new ticket closed and this one reopened (assuming the error on my side is not me missing something).