build a CIRCUITPY_ERASER variant

[dhalbert]

For each board, this builds one special build that erases CIRCUITPY when loaded. The status LED starts at red and goes to blue when done. The build does nothing after erasing: no USB mounting, etc. So it's language-independent.

The GitHub Actions build will create a new CIRCUITPY_ERASER directory in S3 for each board, alongside the language directories.

Basically, this is a simple way of making per-board flash erasers without involving Arduino, etc.

Tested on a Trinket M0, but I need to see GitHub Actions working properly too.

[hierophect]

Thanks for making this. One alternative that I've used for scenarios that needed this is to instruct Circuitpython to enter the repl first, before executing any user code. Simply erasing without having the user do anything is conveniently fire-and-forget, but it's another option to consider - sitting in the REPL would let you access the filesystem and extract things you want from it before erasing via storage. This would be done by changing bool skip_repl = true; to false in main().

[dhalbert]

Thanks for making this. One alternative that I've used for scenarios that needed this is to instruct Circuitpython to enter the repl first, before executing any user code. Simply erasing without having the user do anything is conveniently fire-and-forget, but it's another option to consider - sitting in the REPL would let you access the filesystem and extract things you want from it before erasing via storage. This would be done by changing bool skip_repl = true; to false in main().

We had a Slack discussion this morning where I proposed a safe-mode-only variant build, but it would be English-only, so we wouldn't double the number of builds. English-only was not favored, so I just made an eraser, which doesn't have any language to worry about. Obviously it's silly there's a lot of dead code in the firmware, but I started to make another executable that was much smaller and it was a lot more trouble than it was worth.

[hierophect]

@dhalbert makes sense, sounds good to me

[dhalbert]

I'm making this draft while testing builds, so don't bother to review the actual code yet. The builds are not right: it did not do enough of a rebuild after the previous language (CIRCUITPY_ERASE is the same as zh_latn_pinyin). I know how to force a full rebuild but maybe there is something that will take less time.

[tannewt]

I really don't like this. It has nothing to do with CircuitPython. I'd much rather factor out the useful bits from CircuitPython so that they can be used for other cross-board software. I understand this is the more expedient option but I think the extra work put into splitting useful things out will be more useful than we think.

It also erases all of the filesystem when we just want to disable boot.py and code.py to recover from USB mis-settings. This is what safe-mode is for and if it's broken on some boards, then we should fix it.

[hierophect]

It also erases all of the filesystem when we just want to disable boot.py and code.py to recover from USB mis-settings. This is what safe-mode is for and if it's broken on some boards, then we should fix it.

Is there a way to force entry to safe mode from the hardware? I've had to build versions of Circuitpython like this (or revert to old builds of Circuitpython that don't support what I'm working on) for peripheral-related bugs triggered by code.py that hang without crashing into safe mode, or occasionally for code that does something like restarting too fast, like a code.py that just calls microcontroller.reset() right away or something. So I think there's a use case outside USB. But I do get your argument that it isn't really Circuitpython.

[dglaude]

Not sure if it has to be in or out of CircuitPython, but I believe Adafruit should be providing a safe UF2 file to restore the board to a working state, or one to test the board outside of user error...
If done here, it will benefit all of the non Adafruit board too, and every new board will automatically benefit from it.

[Neradoc]

It also erases all of the filesystem when we just want to disable boot.py and code.py to recover from USB mis-settings. This is what safe-mode is for and if it's broken on some boards, then we should fix it.

Boards without a status LED are an issue. Plus the Raspberry pico doesn't even have a reset button, so you couldn't do anything without adding wires. The only "safe" state the user can achieve with a naked pico is bootloader.
There is the nuke UF2 of course, but...

Is it possible for those cases to have a UF2 that just writes some configuration bits that Circuitpython can then read ? Like "Drop this UF2 to start in safe mode". And do it again to restore normal behavior.

[tannewt]

It also erases all of the filesystem when we just want to disable boot.py and code.py to recover from USB mis-settings. This is what safe-mode is for and if it's broken on some boards, then we should fix it.

Is there a way to force entry to safe mode from the hardware? I've had to build versions of Circuitpython like this (or revert to old builds of Circuitpython that don't support what I'm working on) for peripheral-related bugs triggered by code.py that hang without crashing into safe mode, or occasionally for code that does something like restarting too fast, like a code.py that just calls microcontroller.reset() right away or something. So I think there's a use case outside USB. But I do get your argument that it isn't really Circuitpython.

On most boards you can click reset during a 700ms window at startup to restart into safe mode. On the ESP32-S2 the BOOT pin might work that way instead.

Boards without a status LED are an issue. Plus the Raspberry pico doesn't even have a reset button, so you couldn't do anything without adding wires.

I think it's very rare to not have any status LED and even without it, the safe mode click should work (when a button exists.) We should fix the Pico to use the BOOTSEL state.

[dhalbert]

I did start to implement this as a separate small executable based on the circuitpython code base (filesystem_Init() and the status LED), but it was becoming painful, and I realized the infrastructure I needed was right there in the existing code and build system. It creates a version for each board with little trouble. Yes, it is expedient; we need something soon. It's not elegant, but it works.

This could easily be changed to just rename boot.py and code.py, and do nothing else (and reformat CIRCUITPY if the filesystem can't be found because it is corrupt).

The 700ms safe mode window is hard to hit. My reaction time is not that good, and I have resorted to explaining this as a "slow" double-click, since I can't wait for yellow and then click again. Maybe just lengthening it slightly would be helpful. I would love an easier way to get into safe mode.

In the long run a triple-click in the bootloader to enter safe mode or something similar would be nice, but it won't work on all boards.

[dhalbert]

I'd say we could use one more of the following:

1. CIRCUITPY eraser - the CircuitPython build system knows the board configurations so that's why I based this on CPy. A "board database" would be nice in the long run, but it's not holding up progress. This requires only one build per board because it is not translation-dependent.
2. A safe-mode-only build. This requires doubling the number of artifacts, which will significantly increase CI time. I would rather not do that.
3. A build that does nothing but rename boot.py and code.py so that they will not run. It does not need to go to the REPL or similar, so it can be language-independent, so one build per board.
4. Make getting into safe mode easier. Maybe write up some language about how to do a "slow double-click" using the current 700ms timeout, or lengthen the timeout slightly. For boards that don't have the safe-mode click, fix that. Any trickier way of getting into safe mode like grounding pins, etc. will not work for projects that are in enclosures, etc.

[tannewt]

The 700ms safe mode window is hard to hit. My reaction time is not that good, and I have resorted to explaining this as a "slow" double-click, since I can't wait for yellow and then click again. Maybe just lengthening it slightly would be helpful. I would love an easier way to get into safe mode.

I'm totally open to increasing the time. It only occurs on power up so it's ok to make it longer. It's actually a hair under 700ms because it's 700 ticks which are just under a ms each.

2. A safe-mode-only build. This requires doubling the number of artifacts, which will significantly increase CI time. I would rather not do that.

I think we could patch builds instead of rebuilding to do this. (Have a variable to check for safe mode and use the map file to know where it is in the binary and then change it.) It'd still double the number of artifacts but not the build time.

[dhalbert]

Closing for now.

[hierophect]

I think we could patch builds instead of rebuilding to do this. (Have a variable to check for safe mode and use the map file to know where it is in the binary and then change it.) It'd still double the number of artifacts but not the build time.

I like this idea - if it works, it achieves all the objectives in this thread, without the CI drawback.

[rjsdotorg]

I'd say we could use one more of the following:

1. CIRCUITPY eraser - the CircuitPython build system knows the board configurations so that's why I based this on CPy. A "board database" would be nice in the long run, but it's not holding up progress. This requires only one build per board because it is not translation-dependent.

Is there a how-to for this yet? I think anything that can be installed should have an uninstaller
We have a Metro M0 board that we cannot get to load an Arduino sketch - ie, we want to remove OEM CircuitPython.
We have tried the power-up and double-click etc on learn.adafruit.com and simply have gotten various errors upon upload failure.
Same system loads same sketches on Adalogger, XBoard etc.

In short, how to nuke/remove CircuitPython and leave it in a state for Arduino IDE?

[deshipu]

@rjsdotorg this bug was about erasing the contents of the filesystem on the flash, not CircuitPython itself. You can erase CircuitPython easily enough by simply flashing any Arduino program.

If your Arduino IDE can't flash it, it's not CircuitPython's fault.

[dhalbert]

@rjsdotorg Please bring this up in https://forums.adafruit.com or in discord (https://adafru.it/discord) and we can help diagnose. There are many possible causes for this.

[rjsdotorg]

@rjsdotorg Please bring this up in https://forums.adafruit.com or in discord (https://adafru.it/discord) and we can help diagnose. There are many possible causes for this.

Will do. Thanks.