"Timed out waiting for packet header" on kernel 5.13.10 (only for CH340 + ESP8266) [fixed in kernels 5.14 & 5.13.14] (ESPTOOL-291) #653

jypma · 2021-08-19T12:39:47Z

I've recently been having problems trying to flash any ESP8266 board that has a CH340 USB on Linux, with kernel 5.13.10. Flashing with kernel 5.10.56-1-lts works fine with the same boards. Flashing using a different USB-to-serial (e.g. cp210x) works fine with the latest kernel, as does flashing an ESP32.

Only an ESP8266 with CH340 doesn't work, which includes my NodeMCU and D1 mini boards.

Operating system

Arch Linux, kernel 5.13.10-arch1-1

Python version

3.9.6

What Chip

ESP8266

What development board or other hardware is the chip attached to

Tried a NodeMCU and a D1 mini (both have CH340 for USB)

Is anything else attached to the development board, except for the serial flasher connections?

Bare boards.

Are you running esptool.py from an IDE such as Arduino or Eclipse?

No IDE

Full esptool.py command line that was run:

esptool.py --chip esp8266 chip_id

Full output from esptool.py (on kernel 5.13.10-arch1-1)

esptool.py v3.1
Found 2 serial ports
Serial port /dev/ttyUSB0
Connecting........_____....._____....._____....._____....._____....._____....._____
/dev/ttyUSB0 failed to connect: Failed to connect to ESP8266: Timed out waiting for packet header
Serial port /dev/ttyS4
Connecting........_____....._____....._____....._____....._____....._____....._____
/dev/ttyS4 failed to connect: Failed to connect to ESP8266: Timed out waiting for packet header

A fatal error occurred: Could not connect to an Espressif device on any of the 2 available serial ports.

I've ran esptool once more with trace information, in case that should be of any help (``):

esptool.py v3.1
Serial port /dev/ttyUSB0
Connecting...TRACE +0.000 command op=0x08 data len=36 wait_response=1 timeout=0.100 data=
    0707122055555555 5555555555555555 | ... UUUUUUUUUUUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    55555555                          | UUUU
TRACE +0.001 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.101 Timed out waiting for packet header
.TRACE +0.051 command op=0x08 data len=36 wait_response=1 timeout=0.100 data=
    0707122055555555 5555555555555555 | ... UUUUUUUUUUUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    55555555                          | UUUU
TRACE +0.000 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.101 Timed out waiting for packet header
.TRACE +0.051 command op=0x08 data len=36 wait_response=1 timeout=0.100 data=
    0707122055555555 5555555555555555 | ... UUUUUUUUUUUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    55555555                          | UUUU
TRACE +0.000 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.101 Timed out waiting for packet header
.TRACE +0.051 command op=0x08 data len=36 wait_response=1 timeout=0.100 data=
    0707122055555555 5555555555555555 | ... UUUUUUUUUUUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    55555555                          | UUUU
TRACE +0.000 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.101 Timed out waiting for packet header
.TRACE +0.051 command op=0x08 data len=36 wait_response=1 timeout=0.100 data=
    0707122055555555 5555555555555555 | ... UUUUUUUUUUUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    55555555                          | UUUU
TRACE +0.000 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.101 Timed out waiting for packet header
.TRACE +1.806 command op=0x08 data len=36 wait_response=1 timeout=0.100 data=
    0707122055555555 5555555555555555 | ... UUUUUUUUUUUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    55555555                          | UUUU
TRACE +0.000 Write 46 bytes:
    c000082400000000 0007071220555555 | ...$........ UUU
    5555555555555555 5555555555555555 | UUUUUUUUUUUUUUUU
    5555555555555555 5555555555c0     | UUUUUUUUUUUUU.
TRACE +0.101 Timed out waiting for packet header
[...]

Full output from esptool.py (on kernel 5.10.56-1-lts)

As you can see, on the LTS kernel connectivity is fine.

esptool.py v3.1
Found 2 serial ports
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 2c:3a:e8:08:f1:a6
Uploading stub...
Running stub...
Stub running...
Chip ID: 0x0008f1a6
Hard resetting via RTS pin...

The text was updated successfully, but these errors were encountered:

radimkarnis · 2021-08-19T12:49:34Z

Hello @jypma,

can you see any output from the chip (e.g. ESP8266 boot log) if you connect with a terminal application (e.g. miniterm)? Please, try baud rates 74880 and 115200. You also may want to press the reset button a few times for anything to appear.

Radim

jypma · 2021-08-19T12:53:55Z

Yes, e.g. on the NodeMCU when I hold "flash" and then press "reset", I get (on 74880 baud)

 ets Jan  8 2013,rst cause:2, boot mode:(1,7)

(that's on kernel 5.13.10-arch1-1)

radimkarnis · 2021-08-19T13:00:20Z

Great! That means the USB-UART communication is working, but there might be a problem with the timing of the DTR and RTS signals. Now, if you put the chip into download mode manually (as you just did by holding "flash" and then pressing "reset") OR if you hold the "flash" button and try esptool.py --chip esp8266 chip_id, does it change the outcome?

jypma · 2021-08-19T13:24:41Z

No change trying holding down flash, nor holding flash and pressing reset. Though, if I mash "flash" and then "reset" repeatedly while esptool connects, I sometimes make the error go to Invalid head of packet (0x72), and in another attempt I made esptool hang completely. But no success either way, unfortunately.

jypma · 2021-08-19T15:54:30Z

I do have a logic analyzer. Would it help if I try to capture a timing diagram? I could try to grab RX, TX and RST (and perhaps GPIO15?), and see what that brings. If I find the time :)

k4r4c · 2021-08-19T20:31:13Z

Hi!
I do have the same output when running esptool.py --chip esp8266 chip_id.
I am on archlinux 5.13.10-arch1-1 with Python 3.9.6 on a NodeMCU, with the latest esptool.py version (v3.2-dev).

jypma · 2021-08-20T05:53:49Z

@k4r4c Thanks for chiming in! What happens for you if you boot from the linux-lts kernel instead?

radimkarnis · 2021-08-20T07:23:55Z

Thank you @jypma for looking into this.
If you have the time, could you please try capturing the timing diagram of RX, TX, RST, and GPIO15?
If you manage to capture the difference occurring between the kernels, we could try to pinpoint the issue.

@k4r4c thanks for the confirmation. can you please specify which USB to serial chip are you using?

jypma · 2021-08-20T12:46:39Z

Here's a timing diagram for the not-working scenario (newest kernel):

jypma · 2021-08-20T12:47:16Z

And here's the diagram for the working scenario (LTS kernel)

jypma · 2021-08-20T12:48:52Z

If I compare the two, it looks like the incoming request to to handshake (on RX) comes too early after the initial boot message (on TX), in the non-working case. This is probably related to the GPIO2 "jiggling" starting ~25ms later in the newest kernel.

k4r4c · 2021-08-20T19:50:15Z

@jypma On linux-lts 5.10.56, with esptool.py v3.2-dev, the problem is gone. I successfully connect and flash the esp.
@radimkarnis It uses the CH340G USB to serial chip.

pauledd · 2021-08-22T06:54:10Z

I can confirm this issue up to linux-5.14-rc5. No flashing possible until I downgrade kernel to 5.10.52.
Bug report opened:
https://bugzilla.kernel.org/show_bug.cgi?id=214131

@jypma I attached your images to the bug report. Sorry for stealing.

jypma · 2021-08-22T08:18:16Z

@pauledd Thanks a lot for forwarding the bug, and no problem :-)

basementmedia2 · 2021-08-24T19:57:40Z

I have the same problem. Is the bug already fixed?
I downgraded my kernel to 5.10.52_rt4 but the upload still does not work.
Do i have also to install a newer version of the esptool.py? I use 3.0?
If yes: How can i install a newer version of esptool.py on manjaro.

i tried

pip install esptool
which gave me the following output

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: esptool in ./.local/lib/python3.9/site-packages (3.1)
Requirement already satisfied: bitstring>=3.1.6 in ./.local/lib/python3.9/site-packages (from esptool) (3.1.9)
Requirement already satisfied: cryptography>=2.1.4 in /usr/lib/python3.9/site-packages (from esptool) (3.4.7)
Requirement already satisfied: ecdsa>=0.16.0 in ./.local/lib/python3.9/site-packages (from esptool) (0.17.0)
Requirement already satisfied: pyserial>=3.0 in /usr/lib/python3.9/site-packages (from esptool) (3.5)
Requirement already satisfied: reedsolo<=1.5.4,>=1.5.3 in ./.local/lib/python3.9/site-packages (from esptool) (1.5.4)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python3.9/site-packages (from ecdsa>=0.16.0->esptool) (1.16.0)

So it seems that i already habe the latest esptool.py Version installed, haven't I?

But when i upload a sketch in Arduino IDE it still shows "esptool.py v3.0"
How can i force Arduino to use version 3.1?

Thank you for your help.
Best wishes

Daniel

radimkarnis · 2021-08-25T14:15:02Z

Hello @basementmedia2,
you do not need to update esptool, as the bug happens in the kernel. Btw, Arduino uses its own executable of esptool and doesn't rely on your instance installed from PyPI.

Your upload problems are probably caused by something else. I recommend going through the troubleshooting guide.

Radim

radimkarnis · 2021-08-25T14:15:28Z

Thank you everyone for your help with investigating this issue! The bug has been identified to be in the kernel. It should be already reverted and backported.

There will be no action taken from the side of esptool. I will leave this issue open until someone confirms this.

ufud-org · 2021-08-31T10:39:45Z

I have just re-compiled my VoidLinux Kernel 5.13 using xbps-src and the patch provided by Johan Hovold and it fixed the problem with my USB ch341 and esptool. It may take some time until this patch is available for the various Linux distros out there, but the fix itself works.

basementmedia2 · 2021-08-31T11:22:04Z

Sound great !

radimkarnis · 2021-09-02T13:21:24Z

Thank you @ufud-org for the confirmation.

Closing this issue, feel free to reopen if you feel like further discussion is needed.

dobairoland · 2021-09-02T13:28:51Z

Do we know in which version did this got fixed? We could update the issue title for future reference.

ufud-org · 2021-09-02T17:10:09Z

Link below is tagged 5.14-rc8 but backport to stable is also being mentioned:

https://git.kernel.org/pub/scm/linux/kernel/git/johan/usb-serial.git/tag/?h=usb-serial-5.14-rc8

basementmedia2 · 2021-09-03T09:29:32Z

Hi, i am a Linux beginner and use Manjaro 21.1.0.
Do i have to wait until a new Manjaro Kernel is released or can i install this fix manually?
How do i get informed when the fix is released Manjaro and will such a bug only be fixed in the newest Kernel or also in older Kernels (e.g. the last Realtime Kernel or the latest LTS kernel)?

Sorry for this maybe stupid beginner querstions ;-(

pfrenssen · 2021-09-03T09:36:21Z

Hey thanks for the question but installing Linux kernels is out of scope for the esptool issue queue. You will get a good answer if you go to the Manjaro forums (https://forum.manjaro.org/), there you can find people who are experts on installing Linux kernels. I can also recommend to read the official Manjaro documentation page (https://wiki.manjaro.org/index.php/Manjaro_Kernels) that explains how to install new kernel versions and choose between them. Good luck!

ttencate · 2021-09-11T07:38:49Z

I can confirm that this has been fixed in kernel 5.14.2-arch1-2, which is the first stable kernel to reach the Arch repositories after the broken one.

According to changelogs, the fix is in 5.13.14 already as well.

dobairoland · 2021-09-13T07:22:56Z

Thanks @ttencate!

basementmedia2 · 2021-09-13T10:49:51Z

Yes, great. Just updated the kernel and it works ;-)

bxparks · 2021-10-04T19:24:12Z

I hit this bug by upgrading to the latest Ubuntu 20.04.3 LTS kernel yesterday:

Linux xxx 5.4.0-88-generic #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Downgrading to the previous kernel version fixes the problem:

Linux xxx 5.4.0-86-generic #97-Ubuntu SMP Fri Sep 17 19:19:40 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

euphi · 2021-10-06T20:30:16Z

FYI: For Ubuntu 21.04, Kernel 5.11.0-36 works fine, while 5.11.0-37 is affected.

keithehenry · 2021-10-16T14:04:28Z

For me, the painless solution was to use "Advanced boot options" (or similar) at boot and select an older kernel. No other mods required.

bxparks · 2021-10-16T16:16:34Z

It's not quite painless on some of recent Ubuntu versions which don't bother showing a boot prompt. There are tons of obsolete search results to sift through to find out how to display that boot prompt.

The biggest challenge, though, is knowing enough to even suspect that this is a Linux kernel issue. It took me about 8 hours of trial and error, to eliminate all other potential problems, before I somehow did a search for a kernel bug.

antoniosap · 2021-11-03T10:46:09Z

downgrading kernel now work
Linux ... 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

lbuque · 2023-02-14T06:03:45Z

I also encountered a similar problem.

Here's some information from my OS:

Python 3.9.5
kernel 5.4.0-74-generic
Linux Mint 20.2 Uma \n \l

esp-idf: e8bdaf91986a41678adc6be13888fc037b1acb68

lewin@mint0:~/workspace/micropython/ports/esp32$ /home/lewin/.espressif/python_env/idf4.4_py3.9_env/bin/python ../../../../esp/esp-idf/components/esptool_py/esptool/esptool.py -p /dev/ttyACM0 -b 115200 --before default_reset --after no_reset --chip esp32s3  write_flash --flash_mode dio --flash_size 16MB --flash_freq 80m 0x0 build-LILYGO_T5-4.7-PLUS/bootloader/bootloader.bin 0x8000 build-LILYGO_T5-4.7-PLUS/partition_table/partition-table.bin 0x10000 build-LILYGO_T5-4.7-PLUS/micropython.bin
esptool.py v3.3.2
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:21:35:04
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00006fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x00173fff...
Compressed 25280 bytes to 15122...
Wrote 25280 bytes (15122 compressed) at 0x00000000 in 0.5 seconds (effective 433.6 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 117...
Wrote 3072 bytes (117 compressed) at 0x00008000 in 0.0 seconds (effective 523.2 kbit/s)...
Hash of data verified.
Compressed 1455232 bytes to 986726...
Wrote 1455232 bytes (986726 compressed) at 0x00010000 in 16.8 seconds (effective 691.7 kbit/s)...

A fatal error occurred: Packet content transfer stopped (received 25 bytes)

Any suggestions are welcome! ! !

radimkarnis · 2023-02-14T07:56:33Z

Hi @lbuque,
this looks like an issue with the S3 in USB-Serial/JTAG mode that has been solved in the new version of esptool. You can try updating it by running pip install esptool==4.5.

lbuque · 2023-02-14T08:08:04Z

@radimkarnis

I tried it and it still fails.

lewin@mint0:~/workspace/micropython/ports/esp32$ esptool.py -p /dev/ttyACM0 -b 115200 --before default_reset --after no_reset --chip esp32s3  write_flash --flash_mode dio --flash_size 16MB --flash_freq 80m 0x0 build-LILYGO_T5-4.7-PLUS/bootloader/bootloader.bin 0x8000 build-LILYGO_T5-4.7-PLUS/partition_table/partition-table.bin 0x10000 build-LILYGO_T5-4.7-PLUS/micropython.bin
esptool.py v4.5
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:21:35:04
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00006fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x00173fff...
Compressed 25280 bytes to 15122...
Wrote 25280 bytes (15122 compressed) at 0x00000000 in 0.3 seconds (effective 644.2 kbit/s)...

A fatal error occurred: Packet content transfer stopped (received 25 bytes)

jypma added the operation-failure label Aug 19, 2021

github-actions bot changed the title ~~"Timed out waiting for packet header" on kernel 5.13.10+ (only for CH340 + ESP8266)~~ "Timed out waiting for packet header" on kernel 5.13.10+ (only for CH340 + ESP8266) (ESPTOOL-291) Aug 19, 2021

radimkarnis closed this as completed Sep 2, 2021

dobairoland changed the title ~~"Timed out waiting for packet header" on kernel 5.13.10+ (only for CH340 + ESP8266) (ESPTOOL-291)~~ "Timed out waiting for packet header" on kernel 5.13.10 (only for CH340 + ESP8266) [fixed in kernel 5.14] (ESPTOOL-291) Sep 2, 2021

dobairoland mentioned this issue Sep 7, 2021

Automatic reset not working. Only able to connect with --before no_reset (ESPTOOL-300) #656

Closed

dobairoland changed the title ~~"Timed out waiting for packet header" on kernel 5.13.10 (only for CH340 + ESP8266) [fixed in kernel 5.14] (ESPTOOL-291)~~ "Timed out waiting for packet header" on kernel 5.13.10 (only for CH340 + ESP8266) [fixed in kernels 5.14 & 5.13.14] (ESPTOOL-291) Sep 13, 2021

This was referenced Sep 13, 2021

Added modifiable number of sync attempts. (ESPTOOL-301) #657

Closed

esptool.py not detecting hardware via USB (ESPTOOL-306) #660

Closed

aivarannamaa mentioned this issue Sep 21, 2021

Cannot connect to FireBeetle thonny/thonny#1964

Closed

dobairoland mentioned this issue Oct 4, 2021

will not write to esp on Kubuntu 20.04 (ESPTOOL-320) #670

Closed

dobairoland pinned this issue Oct 5, 2021

Spirituss mentioned this issue Oct 9, 2021

Update via USB cable doesn't work after OS/Hassio update esphome/issues#2400

Open

dobairoland mentioned this issue Oct 22, 2021

esptool.FatalError: Failed to connect to ESP8266: Timed out waiting for packet header (ESPTOOL-332) #681

Closed

ExSign mentioned this issue Oct 22, 2021

"Timed out waiting for packet header" still/again present on kernel 5.14.14 (ESPTOOL-335) #683

Closed

hupiv mentioned this issue May 7, 2022

"Timed out waiting for packet header" on OSX BIG SUE for CH340 + ESP8266 (ESPTOOL-450) #737

Closed

1 task

dobairoland unpinned this issue Feb 22, 2023

"Timed out waiting for packet header" on kernel 5.13.10 (only for CH340 + ESP8266) [fixed in kernels 5.14 & 5.13.14] (ESPTOOL-291) #653

"Timed out waiting for packet header" on kernel 5.13.10 (only for CH340 + ESP8266) [fixed in kernels 5.14 & 5.13.14] (ESPTOOL-291) #653

Comments

jypma commented Aug 19, 2021

Operating system

Python version

What Chip

What development board or other hardware is the chip attached to

Is anything else attached to the development board, except for the serial flasher connections?

Are you running esptool.py from an IDE such as Arduino or Eclipse?

Full esptool.py command line that was run:

Full output from esptool.py (on kernel 5.13.10-arch1-1)

Full output from esptool.py (on kernel 5.10.56-1-lts)

radimkarnis commented Aug 19, 2021

jypma commented Aug 19, 2021 • edited Loading

radimkarnis commented Aug 19, 2021

jypma commented Aug 19, 2021

jypma commented Aug 19, 2021 • edited Loading

k4r4c commented Aug 19, 2021

jypma commented Aug 20, 2021

radimkarnis commented Aug 20, 2021

jypma commented Aug 20, 2021

jypma commented Aug 20, 2021

jypma commented Aug 20, 2021

k4r4c commented Aug 20, 2021

pauledd commented Aug 22, 2021 • edited Loading

jypma commented Aug 22, 2021

basementmedia2 commented Aug 24, 2021 • edited Loading

radimkarnis commented Aug 25, 2021 • edited Loading

radimkarnis commented Aug 25, 2021

ufud-org commented Aug 31, 2021

basementmedia2 commented Aug 31, 2021

radimkarnis commented Sep 2, 2021

dobairoland commented Sep 2, 2021

ufud-org commented Sep 2, 2021 • edited Loading

basementmedia2 commented Sep 3, 2021

pfrenssen commented Sep 3, 2021

ttencate commented Sep 11, 2021

dobairoland commented Sep 13, 2021

basementmedia2 commented Sep 13, 2021

bxparks commented Oct 4, 2021

euphi commented Oct 6, 2021

keithehenry commented Oct 16, 2021

bxparks commented Oct 16, 2021

antoniosap commented Nov 3, 2021

lbuque commented Feb 14, 2023

radimkarnis commented Feb 14, 2023

lbuque commented Feb 14, 2023

jypma commented Aug 19, 2021 •

edited

Loading

jypma commented Aug 19, 2021 •

edited

Loading

pauledd commented Aug 22, 2021 •

edited

Loading

basementmedia2 commented Aug 24, 2021 •

edited

Loading

radimkarnis commented Aug 25, 2021 •

edited

Loading

ufud-org commented Sep 2, 2021 •

edited

Loading