openntpd cannot adjust clock with TIME_ERROR reason

[ilyaevseev]

There is an CentOS 7 host with 4.12 kernel from ELrepo and Openntpd 6.2p3.

/var/log/messages is filled by tons of messages:

openntpd: adjusting local clock by -XX.YYs

XX.YY is increased slowly but steadily.

Initially detected delta was relatively small (15 seconds), but current value is >350 secs.

strace -p$ntpd_pid says:

adjtimex({modes=ADJ_OFFSET|ADJ_NANO|0x8000,
offset=-351879120,
freq=32768000,
maxerror=16000000,
esterror=16000000,
status=STA_UNSYNC,
constant=2,
precision=1,
tolerance=32768000,
time={1535711264, 1056},
tick=10000,
ppsfreq=0, jitter=0, shift=0, stabil=0, jitcnt=0, calcnt=0, errcnt=0, stbcnt=0, tai=0
}) = 5 (TIME_ERROR)

It seems that ntpd_adjtime calls adjtime() from glibc, and adjtime calls adjtimex() and ignores positive return values.

So adjtimex() returns TIME_ERROR to adjtime(), but openntpd dont knows about that because adjtime() returns 0.

It would be useful to change ntpd_adjtime() behaviour:

• call adjtimex() instead of adjtime()

• handle TIME_ERROR result -- for example, decrease requested delta and call adjtimex() again:

  do {
      d_to_tx(d, &tx);
      rc = adjtimex(&tx);
  } while(rc == TIME_ERROR && (d /= 2) > (LOG_NEGLIGIBLE_ADJTIME / 1000));
  
  if (rc == TIME_ERROR) {
      log_warn("adjtimex returned TIME_ERROR");
  } else if (rc < 0) {
      log_warn("adjtimex failed");
  }
  

[ilyaevseev]

The problem still exists with openntpd-6.2p3, but not happen with Chronyd, so I'll switch to it.

$ strace -p$(pgrep chronyd)
....
adjtimex({modes=ADJ_FREQUENCY|ADJ_TICK, offset=0, freq=3207388, maxerror=548036, esterror=3174, status=0, constant=2, precision=1, tolerance=32768000, time={1540323019, 971946}, tick=9998, ppsfreq=0, jitter=0, shift=0, stabil=0, jitcnt=0, calcnt=0, errcnt=0, stbcnt=0, tai=0}) = 0 (TIME_OK)
adjtimex({modes=ADJ_FREQUENCY|ADJ_TICK, offset=0, freq=3207207, maxerror=548036, esterror=3174, status=0, constant=2, precision=1, tolerance=32768000, time={1540323019, 971982}, tick=9998, ppsfreq=0, jitter=0, shift=0, stabil=0, jitcnt=0, calcnt=0, errcnt=0, stbcnt=0, tai=0}) = 0 (TIME_OK)
adjtimex({modes=ADJ_MAXERROR|ADJ_ESTERROR|ADJ_STATUS, offset=0, freq=3207207, maxerror=25834, esterror=3056, status=0, constant=2, precision=1, tolerance=32768000, time={1540323019, 972012}, tick=9998, ppsfreq=0, jitter=0, shift=0, stabil=0, jitcnt=0, calcnt=0, errcnt=0, stbcnt=0, tai=0}) = 0 (TIME_OK)

[bcook-r7]

Thanks for the report, I'll keep it open for when we get some time to address it.

[nerzhul]

same issue on our side, we are gathering metrics with node exporter and using openntpd. The alerting rules reports that time is unsync whereas it is

[busterb]

Hi, I released OpenNTPD 6.8p1 yesterday, which at least in local testing appears to have finally fixed this problem. I'd love some reports on this issue being resolved or still existing in other environments.

[LynxChaus]

Much better - now it correct 500ms in 10 minutes.. Correction -94 seconds skew will take ~31 hours.
After reducing AUTO_THRESHOLD to 15 second, ignoring offset sign in handle_auto(), repair lost peer trust inheritance in DNS resolver - ntpd correctly sync clocks on startup. Thnx.