Flashing ESP8266 on OSX High Sierra #3711

codmpm · 2017-10-11T22:23:16Z

Hi guys,

I have a problem uploading code via the Arduino IDE with my normal ESP8266 (ESP-12F, https://github.com/codmpm/esp8266-proto-board) board together with an FTDI adaptor on OSX High Sierra.
Arduino IDE is 1.8.5, as the older versions don't show items in the menu bar. I'm using ESP8266-Arduono-Core, current master from github.

I can use esptool.py 2.0.1 without any problem to upload binaries. But if I use the Arduino IDE (did python get.py again) I got the following error:

esptool v0.4.9 - (c) 2014 Ch. Klippel <ck@atelier-klippel.de>
Beim Hochladen des Sketches ist ein Fehler aufgetreten
	setting board to ck
	setting baudrate from 115200 to 115200
	setting port from /dev/tty.usbserial to /dev/cu.usbserial-AL009RYR
	setting address from 0x00000000 to 0x00000000
	espcomm_upload_file
	espcomm_upload_mem
opening port /dev/cu.usbserial-AL009RYR at 115200
	tcgetattr
	tcsetattr
	serial open
opening bootloader
resetting board
trying to connect
	espcomm_send_command: sending command header
	espcomm_send_command: sending command payload
trying to connect
	espcomm_send_command: sending command header
	espcomm_send_command: sending command payload
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
	espcomm_send_command: receiving 2 bytes of data
Uploading 243504 bytes from /var/folders/cw/ckgc0g2j0qz6nn7djlc_37qr0000gn/T/arduino_build_979142/Blink.ino.bin to flash at 0x00000000
	erasing flash
	size: 03b730 address: 000000
	first_sector_index: 0
	total_sector_count: 60
	head_sector_count: 16
	adjusted_sector_count: 44
	erase_size: 02c000
	espcomm_send_command: sending command header
	espcomm_send_command: sending command payload
	setting timeout 15000
	setting timeout 100
	espcomm_send_command: receiving 2 bytes of data
	writing flash
warning: espcomm_send_command: didn't receive command response
warning: espcomm_send_command(FLASH_DOWNLOAD_DATA) failed
warning: espcomm_send_command: wrong direction/command: 0x01 0x03, expected 0x01 0x04
closing bootloader
error: espcomm_upload_mem failed

("Beim Hochladen des Sketches ist ein Fehler aufgetreten" means "An error occured while uploading the sketch")

I try to upload the standard blink example with the - to me - common settings:

I've also tried the nightly of the arduino IDE but get the same result. Also tripple checked my wiring, but as the upload via esptool.py works without any flaws I think I'm fine with that.

Python Version is 2.7.10...

Any help or ideas are apreciated.

Thank you!

The text was updated successfully, but these errors were encountered:

buchf1 · 2017-10-12T10:23:04Z

I have the same problem. Please let me know if you find any solution.

Thx.

igrr · 2017-10-12T18:00:07Z

Were you able to flash successfully from Arduino IDE before updating OS X?

codmpm · 2017-10-12T18:15:09Z

Yes. No Problems with Arduino 1.8.2 on OSX 10.12 also using github master of the ESP8266 Arduino Core - maybe not the newest version, though.

igrr · 2017-10-12T18:51:33Z

I see, can you please check which FTDI driver is being used now, by running kextstat | grep FTDI in Terminal?

Based on the log, it looks like esptool-ck times out early waiting for the response. It sends FLASH_DOWNLOAD_DATA command, gets no response (espcomm_send_command: didn't receive command response line), then sends FLASH_DOWNLOAD_DONE command and gets response for FLASH_DOWNLOAD_DATA instead (wrong direction/command: 0x01 0x03, expected 0x01 0x04). Which means that the chip did respond, but later than the timeout.
Perhaps some changes in the serial drivers caused the existing code to time out earlier.

I have not yet updated to 10.13 myself because i have some important stuff to do next week, and can't risk breaking the dev environment on the laptop. I hope to have a chance to do the update next week and look into this.

If anyone wants to dive into esptool-ck and debug this, please do :)
Maybe the code can be modified to attempt reading the response several times until an acceptable response is received, as done is esptool.py:
https://github.com/espressif/esptool/blob/293b78102d327da8f627f8abb4d918e24fed326e/esptool.py#L249-L264

codmpm · 2017-10-12T19:32:16Z

Very good point. Here's the output of kextstat:

197    0 0xffffff7f88664000 0x3000     0x3000     com.apple.driver.AppleUSBFTDI (6.0.0) 4AB9F7CD-9185-32E6-8471-053EB5E81C35 <196 99 18 5 4 3 1>

Thank you for your effort.
Sadly I'm not a python person so helping out is difficult on my side.

Silly question: Is it possible to use esptool.py 2.0.1 insteaed of the 0.4.9 version supplied with the esp-core? I've interchanged them but the parameters are completely different...
Or are these two completely different parts?

igrr · 2017-10-12T19:49:38Z

Oh so it's Apple driver after all. Can you please try installing official FTDI driver and doing another test?

…

On Thu, Oct 12, 2017, 15:32 Patrik Mayer ***@***.***> wrote: Very good point. Here's the output of kextstat: 197 0 0xffffff7f88664000 0x3000 0x3000 com.apple.driver.AppleUSBFTDI (6.0.0) 4AB9F7CD-9185-32E6-8471-053EB5E81C35 <196 99 18 5 4 3 1> Thank you for your effort. Sadly I'm not a python person so helping out is difficult on my side. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3711 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJceswYN9TQfhTvFQa5JCqvKOQAlM3Tks5srmlGgaJpZM4P2NXN> .

codmpm · 2017-10-12T19:55:15Z

Sure, will do. Could you point me to the right one? http://www.ftdichip.com/FTDrivers.htm
My guess is the VCP-driver...

I have an FT232RL on my FTDI Adaptor.

igrr · 2017-10-12T20:00:08Z

http://www.ftdichip.com/Drivers/VCP.htm Pick the one for your OS.

…

On Thu, Oct 12, 2017, 15:55 Patrik Mayer ***@***.***> wrote: Sure, will do. Could you point me to the right one? http://www.ftdichip.com/FTDrivers.htm I have an FT232RL on my FTDI Adaptor. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3711 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJcejZFxMSJS-ecPJ8SWAXcdFKIFGQhks5srm6ugaJpZM4P2NXN> .

codmpm · 2017-10-12T20:08:35Z

Hmmm. I've installed the VCP driver 2.4.2 for "Mac OS X 10.9 and above" and did a reboot, but according to kextstat | grep FTDI the apple driver still gets loaded.

As expected it still does not work.

igrr · 2017-10-12T20:33:11Z

That's odd, FTDI driver is supposed to have precedence over Apple's. Have you restarted the computer after installing the driver?
If yes, can you try unplugging the device and running sudo kextunload -b com.apple.driver.AppleUSBFTDI, then plugging device in again? That should cause FTDI driver to be loaded.

codmpm · 2017-10-12T21:00:12Z

I've rebootet, yes.

Also tried to kextunload but after that no driver gets loaded to the kernel extensions. After a reboot com.apple.driver.AppleUSBFTDI gets loaded again.

I understand that this could be a driver issue, but - as said - using esptool.py or somehting like CoolTerm everything works fine. Also, I've never installed an FTDI driver before...

codmpm · 2017-10-12T21:03:06Z

I've also tried to rename the original driver in /System/Library/Extensions/AppleUSBFTDI.kext/Contents/MacOS as suggested for OSX 10.12- but I'm not allowed tomv` the file, even as root.

igrr · 2017-10-12T21:10:44Z

Thanks for all the attempts. I don't think that this really is a driver issue, just esptool-ck is not robust enough with regards to timeouts, and some change in driver behavior probably causes it to fail. Weird that FTDI driver does not want to load though. Maybe they also need to update the driver for 10.13...

…

On Thu, Oct 12, 2017, 17:03 Patrik Mayer ***@***.***> wrote: I've also tried to rename the original driver in /System/Library/Extensions/AppleUSBFTDI.kext/Contents/MacOS as suggested vor OSX10.12- but I'm not allowed tomv` the file, even as root. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3711 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJcehosi04KTFnbMJoKMzS85e-P8N70ks5srn6XgaJpZM4P2NXN> .

codmpm · 2017-10-12T21:28:27Z

That's what I've though cause of the new kext regulations in 10.13.

Thank you for you help and the attempts to solve this. Glad to test something anytime.

igrr · 2017-10-12T21:53:05Z

Oh, and regarding

Silly question: Is it possible to use esptool.py 2.0.1 insteaed of the 0.4.9 version supplied with the esp-core?

These are two completely different applications. Sadly their original authors went for the most generic name possible (esptool), so we add .py and -ck to disambiguate.

esptool.py
esptool-ck

Replacing one with another is not fully straightforward — as you have noticed arguments are different, so the configuration files (boards.txt and platform.txt) need to be adapted to produce the right options for esptool.py. This can certainly be done, for example the ESP32 arduino core is using esptool.py. But as they say in textbooks, this exercise is left to the reader.

codmpm · 2017-10-12T21:57:25Z

Thank you, more things make sense now ;-)

So, is there anything one can do about the problem?

igrr · 2017-10-13T01:31:52Z

Try replacing the final argument with some large timeout, e.g. 5000 (milliseconds) here:
https://github.com/igrr/esptool-ck/blob/638caabc7b70788913df7dd5959636eef7b57e90/espcomm/espcomm.c#L420

Then compile the application (run 'make') and copy/overwrite the esptool you had (hardware/esp8266com/esp8266/tools/esptool/esptool).

codmpm · 2017-10-13T09:15:05Z

I will test this and maybe I should raise an issue over at https://github.com/igrr/esptool-ck/.

Thank you! I will get back if I have something new.

AndreyzZz · 2017-10-14T19:16:41Z

Confirming on clean High Sierra esp8266 couldn't be flashed.
Resolved by:

Disconnect all USB FTDIs;
Install FTDI VCP 2.4.2 (After successful installation go to System Preferences -> Security & Privacy and allow FTDI extension to load);
Reboot, check one more time for Security & Privacy;
Connect FTDI;
run in terminal kextstat | grep FTDI
The output should be like:

 163    0 0xffffff7f8353b000 0x7000     0x7000     com.FTDI.driver.FTDIUSBSerialDriver (2.4.2) E16D9540-AF63-36AD-94C4-B1164FBD6078 <112 52 5 4 3 1>

If the apple FTDI is still being loaded then try installing D2XXHelper.

Still not working?
Remove everything, reboot and try again:

rm -r /Library/Extensions/FTDIUSBSerialDriver.kext
rm -r /Library/Extensions/D2XXHelper.kext
sudo rm -r /Library/Extensions/FTDIUSBSerialDriver.kext/
sudo rm -r /Library/Extensions/D2XXHelper.kext/

igrr · 2017-10-14T21:38:38Z

FWIW I have just tried with Apple's built-in FTDI driver on 10.12, and result is the same as described in this thread. So the issue is Apple's FTDI driver versus official FTDI driver, rather than 10.13 versus 10.12.

…

On Sun, Oct 15, 2017 at 3:16 AM, AndreyzZz ***@***.***> wrote: Confirming on clean High Sierra esp8266 couldn't be flashed. Resolved by: 1. Disconnect all USB FTDIs; 2. Install FTDI VCP 2.4.2 (After successful installation go to System Preferences -> Security & Privacy and allow FTDI extension to load); 3. Reboot, check one more time for Security & Privacy; 4. Connect FTDI; 5. run in terminal kextstat | grep FTDI The output should be like: 163 0 0xffffff7f8353b000 0x7000 0x7000 com.FTDI.driver.FTDIUSBSerialDriver (2.4.2) E16D9540-AF63-36AD-94C4-B1164FBD6078 <112 52 5 4 3 1> 1. If the apple FTDI is still being loaded then try installing D2XXHelper. Still not working? Remove everything, reboot and try again: rm -r /Library/Extensions/FTDIUSBSerialDriver.kext rm -r /Library/Extensions/D2XXHelper.kext sudo rm -r /Library/Extensions/FTDIUSBSerialDriver.kext/ sudo rm -r /Library/Extensions/D2XXHelper.kext/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3711 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJcej18YiPMu2Fm50t3VC6wzm4yAUxBks5ssQifgaJpZM4P2NXN> .

igrr · 2017-10-14T21:50:35Z

By the way, just tested my own proposal about increasing timeouts (#3711 (comment)) and it indeed allows uploading with Apple's FTDI driver. Tested on 10.12 only but I assume that on 10.13 the story is the same.

Helps with writes timing out when Apple's FTDI driver is used esp8266/Arduino#3711

igrr · 2017-10-15T05:16:02Z

Prepared a new release of esptool-ck, which should fix this issue. Please give it a try:
https://github.com/igrr/esptool-ck/releases/tag/0.4.12

codmpm · 2017-10-16T16:41:13Z

Can confirm, esptool-ck 0.4.12 working on High Sierra (10.13) with Arduino 1.8.5 using com.apple.driver.AppleUSBFTDI (6.0.0).

I've simply exchanged esptool-ck in hardware/esp8266com/esp8266/tools/esptool with version 0.4.12

On Sierra (10.12) everything worked for me with the apple-FTDI-driver an esptool-ck 0.4.9.

//edit:
Only working with 115200bps and 230400bps upload speed. Uploading with 460800bps only works in 1 of 5 times.

codmpm · 2017-10-16T17:37:12Z

@AndreyzZz can you explain how to allow loading of the installed FTDI driver? Can't find anything under Preferences -> Security.

AndreyzZz · 2017-10-17T16:17:41Z

@codmpm
After installing the driver it wasn't loaded by the system. So, I tried to load it manually:

sudo kextload -b /Library/Extensions/FTDIUSBSerialDriver.kext

And looked into /var/logs/system.log
There was an error: kext couldn't be loaded "not privileged"
After that I went into security panel and saw that it requires user acceptance to continue. It just a regular message as if you were installing application from .dmg that wasn't signed by apple.

codmpm · 2017-11-15T00:02:53Z

I've exchanged esptool in /Users/<user>/Documents/Arduino/hardware/esp8266com/esp8266/tools/esptool as I've installed the current github version in the user directories of Arduino.

I think if you use the boardmanager to install the ESP environment this should be the same version.

I'm still using the apple driver that came with High Sierra.

TS22082 · 2017-11-15T00:06:11Z

I have a fresh install of macOS High Sierra and am running into the same error messages. I downloaded and ran esptool-0.4.12-osx.tar.gz and this is what I got back:

Thomass-MacBook-Pro:~ thomassmith$ /Users/thomassmith/Downloads/esptool-0.4.12-osx/esptool ; exit;
error: No arguments given. Use -h for help.
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
[Process completed]

I tried running the blinky program and these errors are still coming up:

An error occurred while uploading the sketch
Uploading 226304 bytes from /var/folders/zf/jp15qc5j6gq41js2sljf_h140000gn/T/arduino_build_684924/testSketch.ino.bin to flash at 0x00000000
warning: espcomm_send_command: didn't receive command response
warning: espcomm_send_command(FLASH_DOWNLOAD_DATA) failed
warning: espcomm_send_command: wrong direction/command: 0x01 0x03, expected 0x01 0x04
error: espcomm_upload_mem failed

Arduino 1.8.5

kextstat | grep FTDI gets me:

164    0 0xffffff7f85b42000 0x3000     0x3000     com.apple.driver.AppleUSBFTDI (6.0.0) 4AB9F7CD-9185-32E6-8471-053EB5E81C35 <163 120 21 5 4 3 1>

TS22082 · 2017-11-15T10:36:09Z

I also downloaded and ran FTDIOSBSerialDriver_v2_4_2.dmg, allowed it to run through security, rebooted my device and Im still getting the same errors.

warning: espcomm_send_command: didn't receive command response
warning: espcomm_send_command(FLASH_DOWNLOAD_DATA) failed
warning: espcomm_send_command: wrong direction/command: 0x01 0x03, expected 0x01 0x04
closing bootloader
error: espcomm_upload_mem failed

I tried connecting an Arduino Uno and the serial output runs fine. Just having issues with the ESP thing dev.

TS22082 · 2017-11-16T17:11:10Z

Hey @igrr just to confirm; You were able to get arduino to upload code to the esp8266 chip after installing esptool-ck? I used Arduino Board manager to setup the board and downloaded the esptool-ck from the link you posted earlier.

igrr · 2017-11-16T17:23:29Z

Could you please enable verbose uploader output (in arduino Preferences window) to verify that you are indeed running esptool 0.4.12?

TS22082 · 2017-11-16T18:52:47Z

High Igrr I'm sorry to waste your time I just enabled a verbose uploader output and to my surprise it worked perfectly. Really not sure what changed, I just slept on it.

d-a-v · 2017-12-13T10:58:36Z

@codmpm @TS22082 is OSX High Sierra with esptool solved ?
@codmpm if so, can you close the issue ?

codmpm · 2017-12-13T12:26:25Z

@d-a-v yes, solved as mentioned in #3711 (comment)

Thank you all!

TS22082 · 2017-12-15T17:18:06Z

@d-a-v solved for me! Thank you.

xelfer · 2017-12-18T22:14:32Z

My esptool was in /Users//Library/Arduino15/packages/esp8266/tools/esptool/0.4.9 and replacing the binary with the one in https://github.com/igrr/esptool-ck/releases/download/0.4.12/esptool-0.4.12-osx.tar.gz fixed my high sierra issues. Thanks!

In case anyone is googling, this was the error I was getting:
warning: espcomm_sync failed
error: espcomm_open failed
error: espcomm_upload_mem failed

richardvk · 2017-12-29T04:55:09Z

I had the same issue (errors below), on Mac High Sierra, and installing the FTDI driver as per igrr solved the problem:

http://www.ftdichip.com/Drivers/VCP.htm

warning: espcomm_send_command: didn't receive command response
warning: espcomm_send_command(FLASH_DOWNLOAD_DATA) failed
An error occurred while uploading the sketch
warning: espcomm_send_command: wrong direction/command: 0x01 0x03, expected 0x01 0x04
error: espcomm_upload_mem failed

sabag · 2018-03-08T06:43:56Z

It worked for me !! thanks a lot.
story: I had Sierra and flashing the ESP12F was fine , once i updated to "High Sierra" it stopped working with the output described above. but once i installed the VCP driver mentioned by @igrr , unplugged, reboot and arduino flashing started to work fine.
so thank you :-)

darrenrector91 · 2019-01-31T21:42:44Z

Download the driver here and install on your Mac. I'm using a 2017 MacBook Pro and after installing I restarted the computer and once restarted I opened the Arduino IDE and went to ports and selected the UART port for the ESP8266. If you don't see the port available in the list then go to your terminal and type system_profiler SPUSBDataType and hit enter and you should see the print out below `USB 3.0 Bus

  Host Controller Driver: AppleUSBXHCISPTLP
  PCI Device ID: 0x9d2f 
  PCI Revision ID: 0x0021 
  PCI Vendor ID: 0x8086 

    CP2102 USB to UART Bridge Controller:

      Product ID: 0xea60
      Vendor ID: 0x10c4  (Silicon Laboratories, Inc.)
      Version: 1.00
      Serial Number: 0001
      Speed: Up to 12 Mb/sec
      Manufacturer: Silicon Labs
      Location ID: 0x14300000 / 17
      Current Available (mA): 500
      Current Required (mA): 100
      Extra Operating Current (mA): 0

USB 3.1 Bus:

  Host Controller Driver: AppleUSBXHCIAR
  PCI Device ID: 0x15d4 
  PCI Revision ID: 0x0002 
  PCI Vendor ID: 0x8086 
  Bus Number: 0x00 `

If you don't see this in our terminal then try switching cables. I found that only one of my cables that worked in my MacBook Pro works for me and the esp8266 to get the above print out. When you do see the above print out then go back to your Arduino IDE and look under ports and you should see the /dev/cu.SLAB_USBtoUART port available.

andordavoti · 2020-06-10T21:08:44Z

Have someone tried installing the FTDI driver on macOS Catalina? It says it isn't compatible with the current version of macOS. Is there any workaround?

codmpm changed the title ~~Flashin ESP8266 on OSX High Sierra~~ Flashing ESP8266 on OSX High Sierra Oct 11, 2017

igrr added a commit to igrr/esptool-ck that referenced this issue Oct 14, 2017

serialport: increase default read timeout

1bd2729

Helps with writes timing out when Apple's FTDI driver is used esp8266/Arduino#3711

igrr added component: tools staged-for-release type: bug labels Oct 15, 2017

igrr added this to the 2.4.0 milestone Oct 15, 2017

codmpm closed this as completed Dec 13, 2017

igrr removed the staged-for-release label Jan 2, 2018

jannnfe mentioned this issue Feb 15, 2018

1 Channel Inching/Self-lockIing Mode WiFi Wireless Switch 5V/12V and Sonoff-Tasmota? arendst/Tasmota#893

Closed

ajitam mentioned this issue Feb 16, 2018

Esp reset Edzelf/Esp-radio#120

Open

Philipp1887 mentioned this issue Apr 13, 2018

error upload ItKindaWorks/ESP8266#13

Open

Flashing ESP8266 on OSX High Sierra #3711

Flashing ESP8266 on OSX High Sierra #3711

Comments

codmpm commented Oct 11, 2017 • edited

buchf1 commented Oct 12, 2017

igrr commented Oct 12, 2017

codmpm commented Oct 12, 2017

igrr commented Oct 12, 2017

codmpm commented Oct 12, 2017 • edited

igrr commented Oct 12, 2017 via email

codmpm commented Oct 12, 2017 • edited

igrr commented Oct 12, 2017 via email

codmpm commented Oct 12, 2017 • edited

igrr commented Oct 12, 2017

codmpm commented Oct 12, 2017 • edited

codmpm commented Oct 12, 2017 • edited

igrr commented Oct 12, 2017 via email

codmpm commented Oct 12, 2017

igrr commented Oct 12, 2017

codmpm commented Oct 12, 2017 • edited

igrr commented Oct 13, 2017

codmpm commented Oct 13, 2017 • edited

AndreyzZz commented Oct 14, 2017

igrr commented Oct 14, 2017 via email

igrr commented Oct 14, 2017

igrr commented Oct 15, 2017

codmpm commented Oct 16, 2017 • edited

codmpm commented Oct 16, 2017 • edited

AndreyzZz commented Oct 17, 2017 • edited

codmpm commented Nov 15, 2017

TS22082 commented Nov 15, 2017 • edited

TS22082 commented Nov 15, 2017

TS22082 commented Nov 16, 2017

igrr commented Nov 16, 2017

TS22082 commented Nov 16, 2017

d-a-v commented Dec 13, 2017

codmpm commented Dec 13, 2017 • edited

TS22082 commented Dec 15, 2017

xelfer commented Dec 18, 2017 • edited

richardvk commented Dec 29, 2017 • edited

sabag commented Mar 8, 2018 • edited

darrenrector91 commented Jan 31, 2019 • edited

andordavoti commented Jun 10, 2020

codmpm commented Oct 11, 2017 •

edited

codmpm commented Oct 12, 2017 •

edited

codmpm commented Oct 12, 2017 •

edited

codmpm commented Oct 12, 2017 •

edited

codmpm commented Oct 12, 2017 •

edited

codmpm commented Oct 12, 2017 •

edited

codmpm commented Oct 12, 2017 •

edited

codmpm commented Oct 13, 2017 •

edited

codmpm commented Oct 16, 2017 •

edited

codmpm commented Oct 16, 2017 •

edited

AndreyzZz commented Oct 17, 2017 •

edited

TS22082 commented Nov 15, 2017 •

edited

codmpm commented Dec 13, 2017 •

edited

xelfer commented Dec 18, 2017 •

edited

richardvk commented Dec 29, 2017 •

edited

sabag commented Mar 8, 2018 •

edited

darrenrector91 commented Jan 31, 2019 •

edited