Clockdrift in the created guest

[dirkdev98]

Description

Running Lima via Colima (https://github.com/abiosoft/colima/), I have occasional clock drift issues. I am unable to find a cause for this or even a way to reproduce it. It seems to be after I lock my laptop and get back the next morning, but only once in a while (every 10 days or so) and not consistently every day.

Versions:

❯ uname -a
Darwin Dirks-MacBook-Pro.local 21.4.0 Darwin Kernel Version 21.4.0: Mon Feb 21 20:36:53 PST 2022; root:xnu-8020.101.4~2/RELEASE_ARM64_T8101 arm64

❯ lima --version
limactl version 0.9.2

❯ colima version  
colima version 0.3.4
git commit: 5a4a70481ca8d1e794677f22524e3c1b79a9b4ae

runtime: docker
arch: aarch64
client: v20.10.14
server: v20.10.11

docker run -it --rm --cap-add SYS_TIME --privileged -e ALLOW_CIDR=0.0.0.0/0 -p 123:123/udp geoffh1977/chrony

Writing New Config File

2022-05-07T08:00:43Z chronyd version 4.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP -SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 -DEBUG)
2022-05-07T08:00:43Z Could not read valid frequency and skew from driftfile /var/lib/chrony/chrony.drift
2022-05-07T08:00:43Z Initial frequency -100000.000 ppm
2022-05-07T08:00:47Z System's initial offset : 978.140307 seconds slow of true (step)
2022-05-07T08:00:52Z Backward time jump detected!
2022-05-07T08:02:00Z Selected source 162.159.200.1 (pool.ntp.org)
2022-05-07T08:02:00Z System clock wrong by 978.685231 seconds
2022-05-07T08:18:18Z System clock was stepped by 978.685231 seconds
2022-05-07T08:02:57Z Backward time jump detected!
2022-05-07T08:02:57Z Can't synchronise: no selectable sources
2022-05-07T08:05:07Z Selected source 94.198.159.15 (pool.ntp.org)
2022-05-07T08:05:07Z System clock wrong by 977.327132 seconds
2022-05-07T08:21:24Z System clock was stepped by 977.327132 seconds
2022-05-07T08:05:06Z Backward time jump detected!
2022-05-07T08:05:06Z Can't synchronise: no selectable sources

References abiosoft/colima#274, #490

[afbjorklund]

Possibly alpine-lima is missing an NTP server ? I thought that lima was adding a RTC in qemu ?

For Ubuntu (systemd) I think it is being handled by systemd-timesyncd, checking with /dev/rtc0

[jandubois]

Possibly alpine-lima is missing an NTP server ? I thought that lima was adding a RTC in qemu ?

It does indeed (in #490). For some reason I remembered that it would sync time with the host instead. So it looks like the RTC gets out of sync as well.

[jandubois]

Here is how Docker Desktop handles this: Addressing Time Drift in Docker Desktop for Mac.

[vielmetti]

I'm seeing something similar in a different configuration (details at #939).

[AndreasA]

The issue still seems to exist. Any timeframe regarding a solution because recently I ran into that issue upon validating a JWT token (not before / expires etc.). the time difference was actually only about 0.4 seconds but that can be enough for its timestamps to not match anymore and validation to fail.
It also seems to exist upon using macOS HyperKit (new virtualization framework).

using busybox as ntp (as mentioned in the article referenced in the comment #850 (comment)) seems to work best. there still is a bit of drift but it seems to be the lowest one for now.

[outcoldman]

Created 2 VMs yesterday, woke up to way more drammatic shift in time

Wed Oct 19 04:16:54 UTC 2118

and

Wed Dec 15 06:03:40 UTC 2213

[outcoldman]

The worst thing, for some reason I cannot even set the date manually

hwclock --hctosys
hwclock: settimeofday() failed: Invalid argument

date -s '2023-09-01 01:00:00'
date: cannot set date: Invalid argument
Fri Sep  1 01:00:00 UTC 2023

[outcoldman]

To fix this issue. I have rebooted both VMs, that sync the dates, and I did after

timedatectl set-ntp no
apt install ntp

Hope that is going to fix this issue for now.

I am on lima 0.17.2, and using ubuntu 23.04/release-20230810/ubuntu-23.04-server-cloudimg-arm64.img

Running with vz on macOS Sonoma.

[krlmlr]

Working around (on Colima):

colima ssh -- sudo date -s @$(TZ=UTC date "+%s")

It's only accurate to the second, that's enough for my use case.