RFC: Add rtc module

[notro]

This PR adds an rtc module and hooks it up to the time module. It also adds rtc support to ports/atmel-samd.
I haven't looked into how it affects/breaks other ports.

This is the ASF4 change: 0001-Add-hal_calendar.patch
I used start.atmel.com to find the right incantations.

Comments are welcome, so I can get this into shape for inclusion.

Testing

I was suprised that there was no unittest module in CP, I used this one: unittest.py

test_rtc.py

import rtc
import time
import unittest

class RtcTestCase(unittest.TestCase):

    def setUp(self):
        self.rtc = rtc.RTC()
        rtc.set_time_source(self.rtc)

    def _skipIfNotImplemented(self):
        try:
            self.rtc.datetime
        except NotImplementedError:
            self.skipTest('No RTC')

    def _struct_time_assertAlmostEqual(self, t1, t2):
        self.assertEqual(t1.tm_year, t2.tm_year)
        self.assertEqual(t1.tm_mon, t2.tm_mon)
        self.assertEqual(t1.tm_mday, t2.tm_mday)
        self.assertEqual(t1.tm_hour, t2.tm_hour)
        self.assertEqual(t1.tm_min, t2.tm_min)
        self.assertAlmostEqual(t1.tm_sec, t2.tm_sec, delta=1)

    def _struct_time_assertEqual(self, t1, t2):
        self._struct_time_assertAlmostEqual(t1, t2)
        self.assertEqual(t1.tm_sec, t2.tm_sec)

    def test_datetime_get(self):
        self._skipIfNotImplemented()
        lt0 = self.rtc.datetime
        lt1 = time.localtime()
        self._struct_time_assertAlmostEqual(lt0, lt1)
        try:
            t0 = time.mktime(lt0)
            t1 = time.mktime(lt1)
            self.assertAlmostEqual(t1, t0, delta=1)
        except OverflowError:
            pass

    def test_datetime_set(self):
        self._skipIfNotImplemented()
        t = time.struct_time((2018, 3, 25, 23, 36, 25, 0, 0, 0))
        self.rtc.datetime = t
        self._struct_time_assertAlmostEqual(time.localtime(), t)

    def test_set_time_source(self):
        rtc.set_time_source(None)
        self.assertRaises(AttributeError, time.localtime)

        t = time.struct_time((2018, 3, 25, 23, 36, 25, 0, 0, 0))

        class RTC(object):
            @property
            def datetime(self):
                return t

        r = RTC()
        rtc.set_time_source(r)
        self._struct_time_assertEqual(r.datetime, t)

        lt = time.localtime()
        self._struct_time_assertEqual(lt, t)
        self.assertEqual(lt.tm_wday, 6)
        self.assertEqual(lt.tm_yday, 84)

test_time.py

# From: https://github.com/python/cpython/blob/master/Lib/test/test_time.py
import time
import unittest

class TimeTestCase(unittest.TestCase):

    def setUp(self):
        self.t = time.time()

    def test_conversions(self):
        self.assertEqual(int(time.mktime(time.localtime(self.t))),
                         int(self.t))

    def test_sleep(self):
        self.assertRaises(ValueError, time.sleep, -2)
        self.assertRaises(ValueError, time.sleep, -1)
        time.sleep(1.2)

    def test_localtime_without_arg(self):
        lt0 = time.localtime()
        lt1 = time.localtime(None)
        t0 = time.mktime(lt0)
        t1 = time.mktime(lt1)
        self.assertAlmostEqual(t1, t0, delta=0.2)

    def test_monotonic(self):
        # monotonic() should not go backward
        times = [time.monotonic() for n in range(100)]
        t1 = times[0]
        for t2 in times[1:]:
            self.assertGreaterEqual(t2, t1, "times=%s" % times)
            t1 = t2

        # monotonic() includes time elapsed during a sleep
        t1 = time.monotonic()
        time.sleep(0.5)
        t2 = time.monotonic()
        dt = t2 - t1
        self.assertGreater(t2, t1)

code.py

import unittest
unittest.main('test_time')
unittest.main('test_rtc')

Problem

The rtc clock is speeding ~0.5 second each day so I need to look into that.

This work started out in #671.

Edit: Update test_rtc.py to use datetime property

[tannewt]

Thank you @notro! I'm pretty busy this weekend but will get back to you tomorrow evening or Monday.

[tannewt]

Overall, this is very exciting! I'm excited to see native time keeping. Thank you for contributing this! I have a few suggestions before we merge though.

Please also make a PR to https://github.com/adafruit/asf4 with the ASF4 patch that you linked to. Once we merge it their, the submodule in this repo can be updated and make Travis happier.

[tannewt]

It might be on by default or done by ASF4.

[tannewt]

Please make this a property. See this for an example: https://github.com/adafruit/circuitpython/blob/master/shared-bindings/digitalio/DigitalInOut.c#L223

Other drivers have this as a property as well: https://github.com/adafruit/Adafruit_CircuitPython_DS3231/blob/master/adafruit_ds3231.py#L111

[tannewt]

Very cool!

[tannewt]

This should be in MP_STATE_VM() so that its considered a root point by the garbage collector. https://github.com/adafruit/circuitpython/blob/master/py/mpstate.h#L171 Without that, we risk cleaning up the time source object before the VM is done.

[tannewt]

You'll want an rtc_reset below as well that sets the time source back to the built-in. Otherwise, you'll refer to an old object when a soft reload happens.

[notro]

How should the other ports that use the time module deal with this rtc module?

1. They should also use the rtc module
2. Add a weak rtc_get_time_source_time() that raises NotImplementedError to keep the time module standalone
3. Add a MICROPY_RTC define that the port can set if it wants/has rtc

[tannewt]

Thanks for the work on this! I'd suggest A variant of 2. that stubs it out in each port. That way its easy to see where to implement it. (Hopefully we'll have it in all that support it eventually.)

Also, you should be able to update this with the newer submodule for asf4 which includes the calendar changes.

Thanks!

[notro]

Version 2

Changes:

• Make RTC.datetime an attribute
• Add RTC.calibration attribute
• Use MP_STATE_VM(rtc_time_source)
• Add rtc_reset()
• Update to latest asf4 commit to get hal_calendar
• Support samd51, only compile tested. I don't know if have got the clock right.
• Add a weak rtc_get_time_source_time() to avoid breaking ports. I just wanted this version out the door (since you already had pulled my asf4 change) so I didn't contemplate your suggestion about a variation of option 2 above. I can look more into that next weekend.

[tannewt]

Thank you so much! I made a couple small tweaks and am merging. Feel free to follow up with any changes you'd still like to make.

[notro]

I see that the switch to using the external oscillator broke various boards and has been reverted.
Suprisingly this didn't affect the accuracy of the Feather M0 Express at all.

I'm getting really nice numbers, but then suddenly something happens that throws it out of wack and 3-4 secs is lost.
So far I haven't been able to find a common factor between runs that can point to a cause (it can take half a day for it to happen).

I'm using these scripts:

• code.py
• On PC: accuracy.py

calibration=-45

First diff:  0.330729 secs

...
-0.043989 secs / 0.372751 days = -0.118013 secs/day (-1.37 ppm) : 22 Apr 2018 00:33:01 (10935.254 - 10915.038 = 20.216)
-0.046051 secs / 0.373446 days = -0.123313 secs/day (-1.43 ppm) : 22 Apr 2018 00:34:01 (10955.495 - 10935.254 = 20.241)
-0.037316 secs / 0.374140 days = -0.099738 secs/day (-1.15 ppm) : 22 Apr 2018 00:35:01 (10975.718 - 10955.495 = 20.223)
-3.624284 secs / 0.374830 days = -9.669146 secs/day (-111.91 ppm) : 22 Apr 2018 00:35:57 (10994.656 - 10975.718 = 18.938)
-3.619775 secs / 0.375524 days = -9.639260 secs/day (-111.57 ppm) : 22 Apr 2018 00:36:57 (11014.875 - 10994.656 = 20.219)
-3.622255 secs / 0.376219 days = -9.628057 secs/day (-111.44 ppm) : 22 Apr 2018 00:37:57 (11035.082 - 11014.875 = 20.207)

Explanation:
One line per minute.
The first number is the time diff between board and local. The date string is from the board. The last numbers are time.monotonic() from the board.
'First diff': I strip off the fact that the clocks don't tick in sync.

Isn't time.monotonic() in seconds? I wonder why that clock only does 20 secs in 60 seconds of wall time.

Adafruit CircuitPlayground Express

I tried this board as well and it is really off, +5 seconds per minute.

I'm currently using this to see if I can calibrate it to something that can be usable:

class RTC(object):
    def __init__(self):
        self._calibration = 0
        self.base = time.mktime(rtc.RTC().datetime)

    @property
    def datetime(self):
        t = rtc.RTC().datetime
        if (abs(self._calibration) > 127):
            secs = time.mktime(t)
            elapsed = secs - self.base
            adjust = float(elapsed) * self._calibration / 10**6
            t = time.localtime(secs + int(adjust))
        return t

    @datetime.setter
    def datetime(self, t):
        rtc.RTC().datetime = t
        self.base = time.mktime(t)

    @property
    def calibration(self):
        return self._calibration

    @calibration.setter
    def calibration(self, val):
        self._calibration = val
        if abs(val) < 128:
            rtc.RTC().calibration = val
        else:
            rtc.RTC().calibration = 0


r = RTC()
rtc.set_time_source(r)

Current run:

calibration=-92900

First diff:  0.760000 secs

...
+5.933410 secs / 0.204905 days = +28.956908 secs/day (+335.15 ppm) : 22 Apr 2018 22:02:07 (15324.073 - 15295.093 = 28.980)
+5.068084 secs / 0.205598 days = +24.650486 secs/day (+285.31 ppm) : 22 Apr 2018 22:03:06 (15353.046 - 15324.073 = 28.973)
+5.208832 secs / 0.206291 days = +25.249978 secs/day (+292.25 ppm) : 22 Apr 2018 22:04:06 (15381.885 - 15353.046 = 28.839)
+5.332829 secs / 0.206984 days = +25.764508 secs/day (+298.20 ppm) : 22 Apr 2018 22:05:06 (15410.765 - 15381.885 = 28.880)
+5.462110 secs / 0.207676 days = +26.301052 secs/day (+304.41 ppm) : 22 Apr 2018 22:06:06 (15439.706 - 15410.765 = 28.941)
+5.596638 secs / 0.208369 days = +26.859217 secs/day (+310.87 ppm) : 22 Apr 2018 22:07:06 (15468.687 - 15439.706 = 28.981)
+5.823488 secs / 0.209073 days = +27.853878 secs/day (+322.38 ppm) : 22 Apr 2018 22:08:07 (15498.120 - 15468.687 = 29.433)
+5.964714 secs / 0.209766 days = +28.435142 secs/day (+329.11 ppm) : 22 Apr 2018 22:09:07 (15527.085 - 15498.120 = 28.965)

I should probably look into calibrating the clock source itself.

[tannewt]

@notro Yup, we had to follow up with a couple fixes. No big deal though. Email me at scott@adafruit.com and I'd be happy to get you more boards to test with if you like. We have an existing bug for supporting external crystals in 3.x and this should be part of it.

Thanks again for the contribution!