Internal RTC has unpredictable time after UF2 load on CP8 espressif

[anecdata]

CircuitPython version

Adafruit CircuitPython 8.0.0-6-g58c0c2212 on 2023-02-14; Adafruit QT Py ESP32S2 with ESP32S2

Code/REPL

# after setting the internal rtc

>>> import time
>>> time.localtime()
struct_time(tm_year=2023, tm_mon=2, tm_mday=15, tm_hour=17, tm_min=4, tm_sec=22, tm_wday=2, tm_yday=46, tm_isdst=-1)
>>> 
>>> import microcontroller
>>> microcontroller.on_next_reset(microcontroller.RunMode.SAFE_MODE)
>>> # or: microcontroller.on_next_reset(microcontroller.RunMode.NORMAL)
>>> microcontroller.reset()

[17:04:54.884] Disconnected
[17:04:55.889] Warning: Could not open tty device (No such file or directory)
[17:04:55.889] Waiting for tty device..
[17:05:08.950] Connected

# after microcontroller.reset into SAFE_MODE or NORMAL

>>> import time
>>> time.localtime()
struct_time(tm_year=2023, tm_mon=2, tm_mday=15, tm_hour=17, tm_min=6, tm_sec=1, tm_wday=2, tm_yday=46, tm_isdst=-1)
>>> 
>>> import microcontroller
>>> microcontroller.on_next_reset(microcontroller.RunMode.UF2)
>>> microcontroller.reset()

[17:07:04.932] Disconnected
[17:07:05.938] Warning: Could not open tty device (No such file or directory)
[17:07:05.938] Waiting for tty device..

# after microcontroller.reset into UF2
#
# ...drop UF2...

[17:09:11.454] Connected

>>> import time
>>> time.localtime()
struct_time(tm_year=1971, tm_mon=3, tm_mday=14, tm_hour=8, tm_min=42, tm_sec=52, tm_wday=6, tm_yday=73, tm_isdst=-1)

Behavior

espressif boards conveniently keep the internal rtc date and time across reloads and software resets (not across power cycles or reset button).

But, recently (8.0.0 beta period), the internal rtc time gets scrambled when a UF2 is loaded, and often it's outside of the range that time.localtime() can process.

CP 7.3.3 starts up after UF2 load with the CircuitPython default time of 2000-01-01 00:00:00.

Description

No response

Additional information

No response

[dhalbert]

To investigate: whether TinyUF2 is causing this problem, and was it a recent TinyUF2 change?

[anecdata]

Does TinyUF2 touch the RTC?

After many rabbit holes in the circuitpython code, I didn't find where the default seconds are set. It seems possible that some code path is missing a check for whether to reset the date, or using an unitialized date to set the RTC.

But with more testing, I did find that the issue trigger is more complex. Hard reset always resets the date to 2000, as expected. CP7 and early CP8 versions will maintain the year 2000 date after a hard reset and through several UF2 loads without another hard reset. Later versions will "randomize" the date on first and subsequent UF2 loads after a hard reset.

However, once the date is bad, CP7 and early CP8 versions will not fix the date through several UF2 loads, they will only maintain a good date. Hard reset is needed to re-establish a good date.

Test procedure:

1. go into UF2 mode
2. load UF2 under test (then wait for reload to finish and enter the REPL)
3. hard reset the device (then wait for reload to finish and enter the REPL)
4. check the date
5. go into UF2 mode
6. reload the UF2 under test (then wait for reload to finish and enter the REPL)
7. check the date

One-liner for checking date & re-entering UF2 mode:

import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()

Bisect yields a cutoff between 8.0.0-beta.6 and 8.0.0-rc.0 for the two behaviors.

beta.6 and before (Good):

Adafruit CircuitPython 8.0.0-beta.6 on 2022-12-21; Adafruit QT Py ESP32S2 with ESP32S2
>>> 
>>> # hard reset...
>>> 

[00:25:25.557] Disconnected
[00:25:26.561] Warning: Could not open tty device (No such file or directory)
[00:25:26.561] Waiting for tty device..
[00:25:28.594] Connected
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 8.0.0-beta.6 on 2022-12-21; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=26, tm_wday=5, tm_yday=1, tm_isdst=-1)

[00:25:53.231] Disconnected
[00:25:54.236] Warning: Could not open tty device (No such file or directory)
[00:25:54.237] Waiting for tty device..
[00:26:18.342] Connected
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 8.0.0-beta.6 on 2022-12-21; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=55, tm_wday=5, tm_yday=1, tm_isdst=-1)

rc.0 and after (Bad):

Adafruit CircuitPython 8.0.0-rc.0 on 2023-01-29; Adafruit QT Py ESP32S2 with ESP32S2
>>> 
>>> # hard reset...
>>> 

[00:15:50.302] Disconnected
[00:15:51.307] Warning: Could not open tty device (No such file or directory)
[00:15:51.308] Waiting for tty device..
[00:15:53.340] Connected
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 8.0.0-rc.0 on 2023-01-29; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=6, tm_wday=5, tm_yday=1, tm_isdst=-1)

[00:15:58.746] Disconnected
[00:15:59.751] Warning: Could not open tty device (No such file or directory)
[00:15:59.751] Waiting for tty device..
[00:16:24.863] Connected
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 8.0.0-rc.0 on 2023-01-29; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=1973, tm_mon=11, tm_mday=29, tm_hour=3, tm_min=52, tm_sec=59, tm_wday=3, tm_yday=333, tm_isdst=-1)

8.0.0 bad date isn't fixed by 7.3.3 without a hard reset:

Adafruit CircuitPython 8.0.0 on 2023-02-06; Adafruit QT Py ESP32S2 with ESP32S2
>>> 
>>> # hard reset...
>>> 

[00:38:16.700] Disconnected
[00:38:17.705] Warning: Could not open tty device (No such file or directory)
[00:38:17.705] Waiting for tty device..
[00:38:19.738] Connected
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 8.0.0 on 2023-02-06; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=6, tm_wday=5, tm_yday=1, tm_isdst=-1)

[00:38:25.121] Disconnected
[00:38:26.123] Warning: Could not open tty device (No such file or directory)
[00:38:26.123] Waiting for tty device..
[00:38:49.209] Connected
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 8.0.0 on 2023-02-06; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=1954, tm_mon=2, tm_mday=24, tm_hour=8, tm_min=54, tm_sec=5, tm_wday=2, tm_yday=55, tm_isdst=-1)

[00:38:56.145] Disconnected
[00:38:57.159] Warning: Could not open tty device (No such file or directory)
[00:38:57.159] Waiting for tty device..
[00:39:41.331] Connected

Adafruit CircuitPython 7.3.3 on 2022-08-29; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=1991, tm_mon=2, tm_mday=5, tm_hour=13, tm_min=14, tm_sec=31, tm_wday=1, tm_yday=36, tm_isdst=-1)

[00:39:49.165] Disconnected
[00:39:50.169] Warning: Could not open tty device (No such file or directory)
[00:39:50.169] Waiting for tty device..
[00:40:15.296] Connected

Adafruit CircuitPython 7.3.3 on 2022-08-29; Adafruit QT Py ESP32S2 with ESP32S2
>>> import time ; time.localtime() ; import microcontroller ; microcontroller.on_next_reset(microcontroller.RunMode.UF2) ; microcontroller.reset()
struct_time(tm_year=1982, tm_mon=3, tm_mday=13, tm_hour=2, tm_min=28, tm_sec=4, tm_wday=5, tm_yday=72, tm_isdst=-1)

[jepler]

might be worth re-checking after we update esp-idf

[anecdata]

This appears to have been fixed somewhere along the way (tested on Adafruit CircuitPython 9.1.0-beta.0 on 2024-03-28; Adafruit QT Py ESP32-S3 4MB Flash 2MB PSRAM with ESP32S3 and Adafruit CircuitPython 9.1.0-beta.0 on 2024-03-28; Adafruit QT Py ESP32S2 with ESP32S2).