ESP32-S2 Upload via USB CDC without Manually Entering Bootloader

[oliverparis]

We have found that we are not able to upload a sketch to an ESP32-S2 via USB CDC without first manually putting the ESP into bootloader mode using PIO. This works as expected in Arduino IDE, but the behaviour is different in PIO.

I've done some investigating and found that the following workaround allows PIO to upload without manually entering bootloader, and I think I can see what's going wrong. First the workaround:

1. Upload a sketch to the ESP which has CDC enabled via any normal method (e.g. the USBSerial.ino example)
2. Remove any specific upload port definition from platformio.ini
3. Run upload of sketch. This will fail, but the ESP will now have been put in bootloader mode
4. Run upload again. This time upload will complete and ESP will reboot into the new sketch.

Why does this work? There are a couple of things happening, I believe starting from the fact that as we know, the ESP is being put into bootloader mode by pulsing the DTR and RTS serial lines. When the ESP goes into bootloader mode, it often enumerates with a different COM port number to what it is on when running CDC in the user's sketch. The PIO implementation (using ESPTool.py) wants you to select a single COM port for upload, so when you run the upload it pulses the DTR and RTS lines and the ESP reboots, but then reappears on a different COM port. Therefore when ESPTool.py tries to find the defined COM port to transfer the .bins it fails.

It appears that Arduino IDE does it slightly differently to get around this. It first pulses the DTS and RTS lines on the chosen CDC COM Port, then actively looks for the new COM port number of the ESP, then launches ESPTool.py upload with that new COM port.

Verbose upload in Arduino IDE gives this output, showing the active search for the new COM port:

...
Forcing reset using 1200bps open/close on port COM20
PORTS {COM1, COM3, COM20, } / {COM1, COM3, COM17, } => {COM17, }
Found upload port: COM17
C:\Users\Oliver\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\3.1.0/esptool.exe --chip esp32s2 --port COM17 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0xe000 C:\Users\Oliver\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.0-rc1/tools/partitions/boot_app0.bin 0x1000 C:\Users\Oliver\AppData\Local\Temp\arduino_build_908897/cdc.ino.bootloader.bin 0x10000 C:\Users\Oliver\AppData\Local\Temp\arduino_build_908897/cdc.ino.bin 0x8000 C:\Users\Oliver\AppData\Local\Temp\arduino_build_908897/cdc.ino.partitions.bin 
esptool.py v3.1
Serial port COM17
Connecting....
Chip is ESP32-S2
...

I realise that this might be unique to the Windows environment due to the way Windows deals with serial ports.

Having, I believe, identified the problem; I have no idea how to go about resolving it. Please can you help?

This investigation came about after an issue was raised on the ESPTinyUSB repo, but we then realised it isn't specifically related to that library.

[calcut]

I am having the same issue on a Mac (so not windows specific).

Pretty sure PIO doesn't do this step
Forcing reset using 1200bps open/close on port /dev/cu.usbmodem1234_56781

Is there a way we can force that?
I notice with the ESP8266 there are configuration options doing something like this.
Perhaps we need something similar for ESP32-S2

https://docs.platformio.org/en/latest/platforms/espressif8266.html#reset-method

[Jason2866]

Note that you may still want to have a boot/reset button (or other method, like a header) on your board, as the DTR/RTS implementation is in software: the USB driver emulates a CDC-ACM serial port and looks at the 'virtual' DTR/RTS lines on that to reset the chip into download mode. If the software crashes the board, or if the user wants to re-purpose the USB peripheral or GPIO pins for something else, there is no way to get into download mode using purely USB anymore. (Also indeed because depending on the software in flash, the first download still needs to have a manual reset, as mentioned above.)

[calcut]

@Jason2866 understood and agreed, but development is a real pain if you have to go to bootloader mode manually every time you change something. I don't need to do that with Arduino, but I do with PIO

[Jason2866]

@calcut yeah, thats why i use for development S2 boards with hardware USB-serial chips on the PCB.
Edit: Have you tried flashing via JTAG? Should work without doing anything manually

[TheNitek]

I am facing the same issue. I am using -D ARDUINO_USB_CDC_ON_BOOT to have Serial mapped to USB. Flashing using esptool works without any hardware buttons pushed, but only if I upload twice (first attempt reboots into DFU, but fails to find the COM port. Second attempt works, since ESP is now in DFU mode).

[tablatronix]

This sucks, what is the solution for this?

@valeros How is this fixed?

[Jason2866]

@tablatronix This is fixed with the commit mentioned.