ntptime.py settime() does not cope with errors

[peterhinch]

This arose in the forum.

If an exception occurs in the time() function, msg is not updated so a value of -NTP_DELTA is returned. If this is the intended mechanism for reporting exceptions, settime() should check for this and avoid updating the RTC.

Or perhaps the finally clause in time() should re-raise the exception?

[Hamsanger]

ntptime() sometimes (randomly) throws the following overflow error:

Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "controller.py", line 200, in RTC_set
  File "ntptime.py", line 38, in settime
OverflowError: overflow converting long int to machine word

Which points to this statement:

    tm = utime.gmtime(t)

I am running on an ESP32 with MicroPython 1.19.1. It has happened twice in two days, but is not easily reproducible. I am using an adjusted NTP_DELTA to account for my timezone (Melbourne), but otherwise just using the built-in ntptime module:

utc_off = 10
dst_off = 0
ntptime.orig_NTP_DELTA = ntptime.NTP_DELTA
ntptime.NTP_DELTA = ntptime.orig_NTP_DELTA - 3600 * (utc_off + dst_off)

[peterhinch]

Unfortunately this has not been fixed. The time() function quite reasonably has a short timeout of 1 sec. However in my testing timeouts do occur occasionally and the code then behaves as described above.

[peterhinch]

I have posted a version which fixes this and other problems with the official version. Documented here.

[Hamsanger]

Thanks for that Peter. It provides a reasonable explanation as to why I was having problems when the internet connection was poor - I will give your version a try.

[Hamsanger]

The version of ntptime.time() provided by @peterhinch (thanks) provides a better basis for a robust NTP time query to mitigate the effect of longer than normal response times. Assuming this is used instead of the packaged version, there are still a couple of failure modes that need to be considered:

• The socket connection may be closed or might close during the timeout wait period due to unreliable WLAN. In this case, the ntptime.time() routine will raise an OSError exception. This should be caught and handled by the caller to prevent a crash.
• If the NTP server is polled too frequently, say several times per minute, it may respond with a Kiss-o'-Death packet to signal that the client must reduce the request rate. (This should not happen in a well behaved application, but could arise in testing, or through user intervention, or if there are multiple devices behind a firewall that all request an NTP time update at a specific time of day). In this case, the time reference value returned is 0 (i.e. the same as the request) and so the ntptime.time() function returns a large negative number i.e. (0 - NTP_DELTA + hrs_offset) which if passed to time.gmtime() will cause an Overflow exception. This can be prevented by simply testing the return value t from ntptime.time() for t > 0, rather than t != 0, before passing the time t to gmtime(). For example, a more robust settime() function would be:

from machine import RTC
import time

def settime(t):
    if t > 0:
        tm = time.gmtime(t)
        RTC().datetime((tm[0], tm[1], tm[2], tm[6] + 1, tm[3], tm[4], tm[5], 0))
        return True
    # else, invalid time value passed in, flag it as ignored    
    return False

The boolean return value from settime() can be tested for success, and if False, the call to ntptime.time() retried after a wait interval of at least 60 seconds (the NTP server's rate limiting mechanism may lock out an ill-behaved requesting client for about a minute to mitigate DOS attacks).

[peterhinch]

Fair points. I'll adapt my version to return 0 on either of those two errors.

Re settime a portable version appears impractical until the RTC class becomes properly portable.

[peterhinch]

@Hamsanger I've pushed an update: ntptime.time() should now return 0 on any error. Perhaps you'd like to review the code. I've tested basic functionality but proving the error handling is an interesting challenge :)

[Hamsanger]

@peterhinch Thanks for the prompt update. The exception handling you have added is more granular than I was using - looks good. I had to re-read the Python documentation to verify the expected behaviour of a return statement in a try block in combination with cleanup code in a finally clause to convince myself it would work as intended, as the out-of-order execution is not intuitive.

I have tested the basic functionality and also field tested with:

• various combinations of network interruption while the code is running (signal dropout and WiFi access point offline)
• shorter socket timeouts (<100msec) to simulate slow responses from the NTP server
• frequent (1/sec) NTP polling to provoke a lock-out response from the NTP server. I get the K-o-D backoff warning and the code handles this gracefully (my previous comments about waiting for >60 seconds to re-try still apply).

All seems to be OK.