Turn on SAMD51 instruction/data cache

[dhalbert]

The SAMD51 has an instruction/data cache. It's off by default, which we didn't realize. The simple patch below to port.c will turn it on.

When it's on, for i in range(n): pass runs 1.6x faster. However, turning it on causes the spi_flash code to loop during CPy start up. If I comment out the filesystem startup code, I can get to the REPL.

So this needs to be added, and then new bugs that crop up will need to be fixed, and we'll need to retest a lot of I/O stuff to find other timing-related bugs.

diff --git a/ports/atmel-samd/supervisor/port.c b/ports/atmel-samd/supervisor/port.c
index 3ac9cbfc9..00ddf2800 100644
--- a/ports/atmel-samd/supervisor/port.c
+++ b/ports/atmel-samd/supervisor/port.c
@@ -90,6 +90,11 @@ safe_mode_t port_init(void) {
     #endif
 #endif
 
+    // Turn on instruction/data cache.
+#ifdef SAMD51
+    hri_cmcc_write_CTRL_reg(CMCC, 1);
+#endif
+

Thanks @WestfW for noticing the equivalent issue for Arduino in adafruit/ArduinoCore-samd#37.

[dhalbert]

Notes on filesystem issues with cache on:

Gets hung up here:

(gdb) bt
#0  0x0002099a in memcpy (dst=dst@entry=0x2002fd24, src=src@entry=0x4000000, n=n@entry=1)
    at ../../lib/libc/string0.c:35
#1  0x000218fc in spi_flash_read_command (command=<optimized out>, command@entry=53 '5', 
    response=response@entry=0x2002fd24 "\310\020", length=length@entry=1) at external_flash/qspi_flash.c:66
#2  0x00021388 in external_flash_init () at external_flash/external_flash.c:229
#3  0x0002146c in external_flash_obj_ioctl (self=<optimized out>, cmd_in=<optimized out>, arg_in=<optimized out>)
    at external_flash/external_flash.c:639
#4  0x0001b35c in disk_initialize (pdrv=<optimized out>) at ../../py/../extmod/vfs_fat_diskio.c:72
#5  0x0001b49a in disk_ioctl (pdrv=<optimized out>, cmd=cmd@entry=5 '\005', buff=buff@entry=0x2002fd67)
    at ../../py/../extmod/vfs_fat_diskio.c:232
#6  0x0001def8 in find_volume (fs=fs@entry=0x200212d0 <fs_user_mount_flash+56>, mode=mode@entry=0 '\000')
    at ../../lib/oofatfs/ff.c:2888
#7  0x0001edde in f_mount (fs=fs@entry=0x200212d0 <fs_user_mount_flash+56>) at ../../lib/oofatfs/ff.c:3112
#8  0x0001c1d6 in filesystem_init (create_allowed=create_allowed@entry=true, force_create=force_create@entry=false)
    at supervisor/filesystem.c:60
#9  0x0001bb12 in main () at ../../main.c:247

I think it's looping around in external_flash/external_flash.c:229 waiting for good status:

    // The suspended write/erase bit should be low.
    do {
        spi_flash_read_command(CMD_READ_STATUS2, read_status_response, 1);
    } while ((read_status_response[0] & 0x80) != 0);

[dhalbert]

I was able to get the QSPI flash to work by turning off the cache and invalidating its entries while inside spi_flash_read_command(). The flow of the SPI flash initialization code is that after reading the JEDEC ID from the SPI flash chip, the SPI flash code then reads the 16-bit SPI flash status register, one byte at a time. But the value it was reading was not the status register but the previously read JEDEC ID, which was stored at the same location, QSPI_AHB. So it appears the previous value (the JEDEC ID) was in the cache. This value was junk in terms of status, and caused the filesystem code to loop in frame number 2 above. So by disabling the cache during command reads, we guarantee to read fresh data.

We are using QSPI in "Serial Memory Mode", which means that the QSPI flash memory is mapped to QSPI_AHB. But the command responses are also mapped there when doing a read command.

I only made that change in spi_flash_read_command(), and yet CIRCUITPY seems to be OK. I need to check if there are other places in the QSPI code to disable the cache to make sure I'm reading fresh data.

There's no discussion in the datasheet of interaction between the CMCC and QSPI peripherals, but there should be, and also nothing in the errata.

I need to understand this more thoroughly before I'm confident this is the fix.

[tannewt]

In addition to the cache disable on QSPI commands we probably want to flush the cache after we write to the QSPI as well. I believe the memory mapping is typically used in a read-only fashion.

[dhalbert]

I am working on setting QSPI.CTRLB.SMEMREG=1 while still having QSPI.CTRLB.MODE=1 (Serial Memory Mode). The datasheet is not quite clear (p. 994) but this seems intended to continue ordinary R/W data access using QSPI_AHB, while allow commands like erase, reset and status get/status to use TXDATA and RXDATA. I think this would allow the cache to remain on because all reads and writes (but not sector erases) would go through the cache.

I have this partly working but it's not clear reads and writes are working properly yet.

[tannewt]

Fixed by #819