ports/mimxrt: Add MSC support.

[iabdalkader]

Summary

Add MSC support using internal flash storage or SD card. This has been sitting for a while in my fork so I figured I should send it upstream. Note this is disabled by default, and can be enabled by boards if needed.

Testing

Tested with:

• MIXRT1020_EVK, Teensy 4.1 and MIMXRT1176_EVK
• RT106x based board.

[github-actions]

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

[robert-hh]

I could test that one. Besides that, I keep an branch for MIMXRT MSC support alive since quite a while. https://github.com/robert-hh/micropython/tree/mimxrt_msc
I had made a PR fro that a while ago, but closed it when USB device support was added to MicroPython in favor of a more general MSC support using Python code, which would be potable across several ports. But that is still to be done.

[robert-hh]

Tried it. At the first run, it did not even compile with a Teensy 4.1 as target after setting MICROPY_HW_USB_MSC, complaining about the symbol MICROPY_HW_SDCARD_SDMMC not being set. Maybe you have that set in your environment. Having MICROPY_HW_SDCARD_SDMMC set, the board creates a new device block device at my PC. That cannot be mounted by littlefs-fuse. The Teensy's file system is littlefs, but a block device should be transparent. Is your PR by some means restricted to FAT?

[iabdalkader]

complaining about the symbol MICROPY_HW_SDCARD_SDMMC not being set. Maybe you have that set in your environment

Yes, this just lets the msc disk know which SD is used for the block device if you have more than one, and since it's board-specific you need to define MICROPY_HW_SDCARD_SDMMC if you enable MSC.

The Teensy's file system is littlefs, but a block device should be transparent. Is your PR by some means restricted to FAT?

You probably need to match the flash sector size, maybe try progsize=4096?

[robert-hh]

Not mounting LFS It was a problem with my PC. After reboot it worked. MICROPY_HW_SDCARD_SDMMC has to be set to 1. Setting it to 2 causes other compile error.

Maybe you get yourself MIMXRT boards and test the submission before posting a PR.

[iabdalkader]

MICROPY_HW_SDCARD_SDMMC has to be set to 1. Setting it to 2 causes other compile error.

Is this unexpected? If you only have one SD card, you can't set the ID to 2.

Maybe you get yourself MIMXRT boards and test the submission before posting a PR.

As I've mentioned I have been using this code in production for year, so it's been tested, just never with an upstream board.

[robert-hh]

I did a few more board tests after changing mpconfigport.h to:

// Enable USB Mass Storage with FatFS filesystem.
#ifndef MICROPY_HW_USB_MSC
#define MICROPY_HW_USB_MSC                  (1)
#if MICROPY_PY_MACHINE_SDCARD && !defined(MICROPY_HW_SDCARD_SDMMC)
#define MICROPY_HW_SDCARD_SDMMC             (1)
#endif
#endif

MIXRT1020_EVK, Teensy 4.1 and MIMXRT1176_EVK worked fine. It is a good idea to expose the inserted SD card.

MIMXRT1010 and MIMXRT1015 fail for two reasons:

1. Overflow of the m_dtcm segment. by 2800 resp 1700 bytes. It may be possible to reorganize the loader file for that,
2. Access to machine_sdcard_type in main.py. These devices do not support SD cards. I've changed that section to:

        #if MICROPY_HW_USB_MSC
        // Set the USB medium to flash block device.
        mimxrt_msc_medium = &mimxrt_flash_type;

        #if MICROPY_PY_MACHINE_SDCARD
        const char *path = "/sdcard";
        // If SD is mounted, set the USB medium to SD.
        if (mp_vfs_lookup_path(path, &path) != MP_VFS_NONE) {
            mimxrt_msc_medium = &machine_sdcard_type;
        }
        #endif
        #endif

[iabdalkader]

@robert-hh I've incorporated both changes. As for the memory overflow, this should be fixed for individual boards if/when they enable this feature.

[robert-hh]

I see that you disable MSC by default.
The memory overflow at least for the 1011 is caused by a buffer with a need 2k alignment, which is placed by the loader just after a 2k boundary. And then there is a ~2k gap. On option would be to reduce the stack size, which is 20k at the moment. The other would be to re-arrange the objects in .bss manually.I know that I did that once, but I forgot how.

[iabdalkader]

I see that you disable MSC by default.

Yes. If you like you can enable it by default or for certain boards if/when this PR gets merged.

As for the memory, I'd lower the stack, 16KBs is good probably good enough.

[robert-hh]

I did both. Rearranging the BSS to reduce gaps and decreasing the stack size.

[dpgeorge]

@robert-hh are you happy with this PR now?

[robert-hh]

Actually the PR still does not build with MIMXRT1011 and MIMXRT1015 devices with MSC enabled. The stack has to be decreased, like from 20k to 16k. Same for MIMXRT1015
@iabdalkader will you do that in mimxrt1011.ld and mimxrt1015.ld? It may be useful when adding a new feature to at least build the firmware for all applicable boards with that feature enabled.

[iabdalkader]

will you do that in mimxrt1011.ld and mimxrt1015.ld? It may be useful when adding a new feature to at least build the firmware for all applicable boards with that feature enabled.

Are you saying we should enable MSC for these boards by default?

[iabdalkader]

@robert-hh Enabled for MIMXRT1011, MIMXRT1015 and MIMXRT1060. I've added 1060 to CI to have at least one board with MSC enabled.

Note I moved MICROPY_HW_SDCARD_SDMMC to msc_disk.c. MICROPY_PY_MACHINE_SDCARD is defined later in the mpconfigport.h so it failed to build if a board disables SD card.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.59%. Comparing base (0a433a0) to head (752c167).
Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #16591   +/-   ##
=======================================
  Coverage   98.59%   98.59%           
=======================================
  Files         167      167           
  Lines       21599    21599           
=======================================
  Hits        21295    21295           
  Misses        304      304           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[robert-hh]

Are you saying we should enable MSC for these boards by default?

Actually no. But if it's disabled, there should be no further hiccup. I enabled MSC by default and ran a build-all script with the 1011/1015 stack changed. All boards build.

The Teensy4.1 board has a MIMXRT1062 MCU, and it is already built in the CI. So you could use that one. But if you want to add a board, the MIMXRT1010 would be a good choice, because that one is always a candidate for trouble.

[dpgeorge]

I don't want to enable MSC by default. It's just that it should build without error if a user enables it themselves.

[iabdalkader]

@dpgeorge You can just drop commit number 3 which enables it for some boards, and commit number 4 which adds 1060 to CI. However, I'd leave 1060 in CI as it tests more things.

[iabdalkader]

I don't want to enable MSC by default. It's just that it should build without error if a user enables it themselves.

@dpgeorge @robert-hh On second thought, if we don't add at least 1 board with MSC enabled, we'll never notice if this ever gets broken by any change. I only care about the core functionality, but this is worth mentioning.

[dpgeorge]

On second thought, if we don't add at least 1 board with MSC enabled, we'll never notice if this ever gets broken by any change

You can build with MSC enabled just under CI using something like make BOARD=... CFLAGS_EXRTA=-DMICROPY_HW_USB_MSC=1.

[iabdalkader]

Done. Seems to build fine.

[dpgeorge]

Tested with TEENSY40 and enabling MSC. It works.

[robert-hh]

As a note: This feature works also with the lfs file system at the board and littlefs-fuse at the PC.

[mattytrentini]

As a note: This feature works also with the lfs file system at the board and littlefs-fuse at the PC.

Oh nice, I've been meaning to try this - that's great to know!

[iabdalkader]

Unrelated question, but does anyone know what the minimum LFSV2 volume size is? Or is there no such limitation ? For example, for FatFS it's 512 * 128 = 64K.

[robert-hh]

Lfs uses at least 5 blocks (2+2+file data), with the minimal block size being 104 bytes. So yes, there is a minimal size, but that is rather small.

[iabdalkader]

Amazing! Thanks @robert-hh

[dpgeorge]

With two blocks of 96 bytes each, it's possible to create a LFS2 filesystem that has one small file.