"Timed out waiting for packet header" - fails at any -b baudrate (ESPTOOL-102)

[gandsnut]

Trying to read firmware off Sonoff device via an FTDI USB-to-serial interface. Lubuntu 18.04
Attempted to use both 2.x and 3.x USB ports, and 2 cables. I have tried the suggested variety of "-b" baud rates from documentation (115200, 921600, 230400, etc).

output from esptool.py:

gandsnut@haserver:/extra/sonoff_specific$ /usr/share/esptool/esptool.py -p /dev/ttyUSB0 -b 12000000 read_flash 0 0x100000 sonoff-original-firmware.bin
esptool.py v2.1
Connecting.....................................................

A fatal error occurred: Failed to connect to Espressif device: Timed out waiting for packet header

hwinfo:

04: USB 00.0: 0700 Serial controller
[Created at usb.122]
Unique ID: UfPf.bzv2XOUTxN6
Parent ID: 2XnU._63LoM0erv2
SysFS ID: /devices/pci0000:00/0000:00:10.0/usb5/5-1/5-1:1.0
SysFS BusID: 5-1:1.0
Hardware Class: unknown
Model: "Future Technology Devices International FT232 Serial (UART) IC"
Hotplug: USB
Vendor: usb 0x0403 "Future Technology Devices International, Ltd"
Device: usb 0x6001 "FT232 Serial (UART) IC"
Revision: "6.00"
Serial ID: "A916MHVD"
Driver: "ftdi_sio"
Driver Modules: "usbserial", "ftdi_sio"
Device File: /dev/ttyUSB0
Device Files: /dev/ttyUSB0, /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A916MHVD-if00-port0, /dev/serial/by-path/pci-0000:00:10.0-usb-0:1:1.0-port0
Device Number: char 188:0
Tags: mouse, modem
Speed: 12 Mbps
Module Alias: "usb:v0403p6001d0600dc00dsc00dp00icFFiscFFipFFin00"
Driver Info #0:
Driver Status: ftdi_sio is active
Driver Activation Cmd: "modprobe ftdi_sio"
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #17 (Hub)

Thanks
rjm@gandsnut.net

[Twilight-Logic]

Similar issue here. I have a LoLin NodeMCU V3 ESP8266 board. The ESP8266 module itself is marked ESP8266MOD.

I am running the ESP8266 community library plugin version 2.6.0 for the Arduino IDE. This comes with esptool.py version 2.7, but I have also tried 2.6.3 with esptool.py version 2.8. I get the following:

Executable segment sizes:
IROM   : 267496          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 26900   / 32768 - code in IRAM          (ICACHE_RAM_ATTR, ISRs...) 
DATA   : 1256  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 17180 ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 33424 )         - zeroed variables      (global, static) in RAM/HEAP 
Sketch uses 312832 bytes (62%) of program storage space. Maximum is 499696 bytes.
Global variables use 51860 bytes (63%) of dynamic memory, leaving 30060 bytes for local variables. Maximum is 81920 bytes.
esptool.py v2.7
Serial port /dev/ttyUSB2
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: cc:50:e3:55:db:52
Uploading stub...
Running stub...
Stub running...
Manufacturer: 20
Device: 4016
Detected flash size: 4MB
Hard resetting via RTS pin...
esptool.py v2.7
Serial port /dev/ttyUSB2
Connecting........_____....._____....._____....._____....._____....._____....._____
Traceback (most recent call last):
  File "/home/johnc/.arduino15/packages/esp8266/hardware/esp8266/2.6.0/tools/upload.py", line 25, in <module>
    esptool.main(fakeargs)
  File "/home/johnc/.arduino15/packages/esp8266/hardware/esp8266/2.6.0/tools/esptool/esptool.py", line 2841, in main
    esp.connect(args.before)
  File "/home/johnc/.arduino15/packages/esp8266/hardware/esp8266/2.6.0/tools/esptool/esptool.py", line 483, in connect
    raise FatalError('Failed to connect to %s: %s' % (self.CHIP_NAME, last_error))
esptool.FatalError: Failed to connect to ESP8266: Timed out waiting for packet header
esptool.FatalError: Failed to connect to ESP8266: Timed out waiting for packet header

In both cases, reverting back to library 2.5.2 which comes with esptool.py version 2.6 works perfectly fine. It would seem that there is some issue with esptool.py 2.7 and 2.8 that is preventing successful uploads.

[Twilight-Logic]

Digging a little further I discovered that esptool.py version 2.7, 2.8 and 3.0 work fine when called from the command line with something like:

./esptool.py --port /dev/ttyUSB2 write_flash 0x000000 mybinary.bin

The problem arises only when esptool.py is called from the Arduino IDE with ESP8266 library versions above 2.5.2, so this may not be a problem with esptool.py, but with the calling application, which in this case is either the Arduino IDE, or the ESP8266 library plugin.

[gandsnut]

@Twilight-Logic : Well, since I'm a __rank__noob, I have to ask: ok, so what's the best way to deal with this? Should I take this to an Arduino IDE forum, or one for the ESP8266 library (presumably here on github)? And for now, once the IDE fails with the observed error, can I use your suggested command line - and if so, can you suggest where I'd find the .bin (in the IDE)? Appreciated!

[Twilight-Logic]

gandsnut, I have no connection with the esptool.py project, but from your comments it sounds like you also encountered the problem while uploading sketches to the ESP8266 with the Arduino IDE?

I have already opened a thread on the Arduino forum about this:
https://forum.arduino.cc/index.php?topic=662043.msg4459413#msg4459413

Please feel free to comment or add any further information to this. I have also opened an issue on the ESP8266 library GitHub:

esp8266/Arduino#7050

I'm not sure if its an Arduino or a ESP8266 library issue, but hopefully bringing attention to it in both places will increase the chances of someone having a look at it.

Exporting a binary version of your sketch from the Arduino IDE is a matter of selecting Sketch | Export compiled binary within the IDE. The exported binary file will be written to the same directory as your sketch but with a .bin extension. With some boards you can end up with two versions of the binary with and without bootloader, and files can also have a .hex suffix, but with the ESP8266 you get just one .bin file. Easiest thing is to simplify the name of the binary and place it in the same folder as the esptool.py program.

You can dig around and locate your copy of esptool.py in the Arduino hierarchy (for my Linux machine it was in /home/user/.arduino15/packages/esp8266/hardware/esp8266/2.6.3/tools/esptool - the version number may vary) but it is probably simpler to just pull the current version from the espressif Github repository into a directory of your choice with:

git clone https://github.com/espressif/esptool.git

By all means use the command line I mentioned above modifying it with the name of your serial port and exported binary, or alternatively for now revert the ESP8266 library back to version 2.5.2 in the IDE by going into Board Manager (Tools | Board: whatever | Board Manager). Search for ESP8266, select version 2.5.2 from the dropdown list and click Install.

Hope this helps.

[gandsnut]

Twilight-Logic: Ok, just read your 2/1/2020 email containing:

"I ran the command again but without the '--before no_reset' parameter:" which you say made the upload succeed. Way cool.

Given this new info, is the (temporary fix) method to find in the Arduino IDE where it sets up each executed line of compilation, then edit out the "--before no_reset" usage? I don't know if that's even possible...

Appreciated (and thanks for the work-around suggestions).

[Twilight-Logic]

Looks like something may have gotten out of sync during the update process. I have just followed a suggestion to delete the ~/.arduino15 directory (I renamed mine) and install from scratch. While this will loose your configured preferences and any additional installed boards, it doesn't take long to set up. At minimum you have to update the AVR library again and install the ESP8266 library. After that the ESP8266 upload worked. If you have any other categories of boards, e.g. Teensy, STM etc then there will likely have to be re-installed as well. Doesn't really explain why and what went wrong but at least it does provide a fix.

UPDATE: a tried a further test with deleting just .arduino15/packages/esp8266. This would avoid loosing preferences and having to re-install other boards. This did work and appears to be sufficient to solve the problem.

[projectgus]

Hi @Twilight-Logic ,

It looks like your problem has been resolved by fixing the bad update of the arduino environment, yes? This is good to hear.

Hi @gandsnut,

Your problem looks different, although the error message is the same.

I would start with the basics: double-check all the connections, check you're correctly setting GPIO0 low to put the Sonoff into bootloader mode (and then re-powering it to reset it), and that TX & RX pins aren't accidentally swapped when connecting to the FTDI board.

[Twilight-Logic]

Hi @Twilight-Logic ,

It looks like your problem has been resolved by fixing the bad update of the arduino environment, yes? This is good to hear.

Yes, seems to be fixed for me. Thanks.

[gandsnut]

@projectgus

I've been diligent in wiring and using reset button to establish bootloader mode. Many lay fault with cheap foreign FTDI boards; apparently the '340' variety are more aggravating. I have yet to try @Twilight-Logic suggestion of reverting to 2.5.2 of the ESP8266 library, though I did try his other suggestion to rename the .arduino15 dir tree and reinstalling. One wonders what repercussions come from that... I did find esptool.py for both 8266 & 32, so I can try the other suggestion to invoke it directly. Somewhere on the 'Net I saw a 'esptool.ck' - written in 'C'? I don't know. I've spent so many hours on this I'm a bit burnt out.

[projectgus]

Hi @gandsnut ,

There is an esptool-ck but if esptool.py can't connect to the chip at all then the problem is almost certainly in the serial port config, the USB/serial adapter board, or the hardware setup on the Sonoff itself.

[luxarts]

Hi, people!

I had the same problem using Arduino IDE 1.8.12 and ESP8266 2.6.3 library. As you mentioned the last version of the library that worked is 2.5.2, which uses esptool v2.6. So I copied the esptool folder from 2.6, installed the ESP8266 2.6.3 library and replaced the esptool version with the 2.6.

1. Install ESP8266 version 2.5.2 from the board manager
2. Go to the folder where the boards are stored and go to esp8266/tools
3. Copy the esptool folder to safe directory
4. Install ESP8266 version 2.6.3 from the board manager
5. Go to esp8266/tools, delete the esptool folder and paste the previous copied one

You should be able to upload sketches using Arduino IDE again.

I'll take a look in the next weeks to the esptool to search for the change that broke the upload.

Hope you find it useful!

[scroggyg]

I am having a problem that I cannot change the baudrate, it is stuck at 460800, I used idf.py menuconfig, saved the changes, and it always stays at 460800. It is changing it in sdkconfig.

[projectgus]

@scroggyg This sounds like an issue with ESP-IDF and not related to esptool at all. Is it possible you set the environment variable IDF_MONITOR_BAUD or MONITORBAUD (old name)? This will override the sdkconfig setting.

(If the above doesn't fix it, please open a new issue on ESP-IDF rather than replying to this unrelated discussion.)

[DuncanWilliamGibbons]

Hi, people!

I had the same problem using Arduino IDE 1.8.12 and ESP8266 2.6.3 library. As you mentioned the last version of the library that worked is 2.5.2, which uses esptool v2.6. So I copied the esptool folder from 2.6, installed the ESP8266 2.6.3 library and replaced the esptool version with the 2.6.

1. Install ESP8266 version 2.5.2 from the board manager
2. Go to the folder where the boards are stored and go to esp8266/tools
3. Copy the esptool folder to safe directory
4. Install ESP8266 version 2.6.3 from the board manager
5. Go to esp8266/tools, delete the esptool folder and paste the previous copied one

You should be able to upload sketches using Arduino IDE again.

I'll take a look in the next weeks to the esptool to search for the change that broke the upload.

Hope you find it useful!

Gave this a try, but still getting the esptool error

[radimkarnis]

Hello @DuncanWilliamGibbons,

Were you able to successfully upload a sketch? Are there any more steps you tried to resolve the esptool error?

[gandsnut]

I've been unable to continue testing this, having temporarily lost the space where I was working. Will have to re-establish my tools and bench at some point in the future. Thanks.

[radimkarnis]

Thank you for the update. I am closing this issue for now, as it seems it is not relevant anymore. Feel free to reopen anytime in the future if you feel further discussion is needed.