esptool fails on MacOS Big Sur preview: Failure to import serial.tools.list_ports

[jeksys]

esptool fails on calling pyserial on MacOS Big Sur with dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found (#540)

esptool.py chip_id

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/bin/esptool.py", line 57, in <module>
    import serial.tools.list_ports as list_ports
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/serial/tools/list_ports.py", line 29, in <module>
    from serial.tools.list_ports_posix import comports
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/serial/tools/list_ports_posix.py", line 31, in <module>
    from serial.tools.list_ports_osx import comports
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/serial/tools/list_ports_osx.py", line 32, in <module>
    kIOMasterPortDefault = ctypes.c_void_p.in_dll(iokit, "kIOMasterPortDefault")
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found

See reported issue for esptool
espressif/esptool#540 (comment)

[ehagerty]

same issue here. NB, putting --port /dev/tty.usbserial-xxxxxxxx flash-id gives the same error. Thanks so much, I should know better than to upgrade my primary machine!

[projectgus]

Hi,

esptool maintainer here.

The underlying issue seems to be that kIOMasterPortDefault is no longer a defined symbol in the iokit library. Possibly it's been moved to a compile-time only constant instead of a library symbol...?

The documentation suggests it's still supported and is synonymous with NULL, so replacing it will NULL (either always or in response to this ValueError) may be a fix? (Or may just result in a more important symbol failing to import.)

I don't have a MacOS install I can check this on right now, but I'm following up with some colleagues who do.

[ehagerty]

I’m a MacOS user with dev tools although I’m not a Mac dev per se - let me know how I can help if that adds value 👍

…

On 27 Jun 2020, at 07:46, Angus Gratton ***@***.***> wrote:  Hi, esptool maintainer here. The underlying issue seems to be that kIOMasterPortDefault is no longer a defined symbol in the iokit library. Possibly it's been moved to a compile-time only constant instead of a library symbol...? The documentation suggests it's still supported and is synonymous with NULL, so replacing it will NULL (either always or in response to this ValueError) may be a fix? (Or may just result in a more important symbol failing to import.) I don't have a MacOS install I can check this on right now, but I'm following up with some colleagues who do. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[RyanMorash]

I tried setting kIOMasterPortDefault to None in the following instance:

pyserial/serial/tools/list_ports_osx.py

Line 34 in 3a6ae5a

kIOMasterPortDefault = ctypes.c_void_p.in_dll(iokit, "kIOMasterPortDefault")

That resulting in the following AttributeError:

Traceback (most recent call last):
  File "/usr/local/bin/esptool.py", line 57, in <module>
    import serial.tools.list_ports as list_ports
  File "/usr/local/lib/python3.7/site-packages/serial/tools/list_ports.py", line 29, in <module>
    from serial.tools.list_ports_posix import comports
  File "/usr/local/lib/python3.7/site-packages/serial/tools/list_ports_posix.py", line 31, in <module>
    from serial.tools.list_ports_osx import comports
  File "/usr/local/lib/python3.7/site-packages/serial/tools/list_ports_osx.py", line 37, in <module>
    iokit.IOServiceMatching.restype = ctypes.c_void_p
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, IOServiceMatching): symbol not found

[RyanMorash]

It looks like this is most likely the issue (the downstream espressif/esptool#540 issue is referenced there) is: https://bugs.python.org/issue41179

From the macOS Big Sur release notes (https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-beta-release-notes):

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

[ghost]

The problem can be "ctypes" lib, for origin...

[dbartelmus]

Download: https://github.com/espressif/esptool/blob/cb2060c0d768da38c9aa463e315e187a91f761f8/esptool.py
And replace your: /Users/USERNAME/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.4/tools/esptool/esptool.py

[sghoweri]

@dbartelmus I tried this esptool.py replacement workaround to no avail 😞 - anyone else running Big Sur manage to get around this issue / error?

[StavrosD]

Download: https://github.com/espressif/esptool/blob/cb2060c0d768da38c9aa463e315e187a91f761f8/esptool.py
And replace your: /Users/USERNAME/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.4/tools/esptool/esptool.py

The patch works, however the path may be different. The actual path of the esptool.py is displayed on the build log.
For me it was:
/Users/(my username)/.platformio/packages/tool-esptoolpy/esptool.py

[projectgus]

Folks, please keep in mind that this issue is on the pyserial repo not the esptool.py repo.

(pyserial is the library that esptool uses to access the serial port, and the error is coming from that library. However the pyserial maintainers aren't expected to know anything about esptool and most of this discussion is totally off topic for pyserial.)

The issue in esptool is here, and a workaround has been committed to the esptool development version already (see linked comment). There is also some discussion in that issue about how to apply this change to Arduino for ESP32:
espressif/esptool#540 (comment)

@jeksys Would you be able to please edit the title of this issue to something like "Failure to import serial.tools.list_ports on MacOS Big Sur preview"? It may help guide esptool users to the other issue.

[projectgus]

@jeksys Sorry, I didn't explain well enough: I mean to remove the mention of "esptool" from the title. Because the problem on the pyserial side is not really to do with esptool, the steps to reproduce are as simple as installing pyserial and then doing run python -c "import serial.tools.list_ports"

[zsquareplusc]

I'm not confused by the esptool, I've actually used it myself ;-)

The issue seems to be the same as #518

[chairman888]

I've put up a PR to address this in pyserial.

[hawtdawg]

Download: https://github.com/espressif/esptool/blob/cb2060c0d768da38c9aa463e315e187a91f761f8/esptool.py
And replace your: /Users/USERNAME/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.4/tools/esptool/esptool.py

Doesn't work for me. I get this error in Arduino:
Traceback (most recent call last): File "/Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 38, in <module> import serial ImportError: No module named serial exit status 1 Error compiling for board ESP32 Dev Module.

Any other things I could try?

[chairman888]

Can you confirm that pyserial is installed one your device?

[hawtdawg]

Can you confirm that pyserial is installed one your device?

Yes. I even updated it just now to be certain. Here's what Terminal says:
User@MBP16 ~ % pip install pyserial Requirement already satisfied: pyserial in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (3.4) User@MBP16 ~ %

[gblach]

Arduino has own copy of Python libraries in different location. I don't have Arduino installed currently, so you must find proper path by yourself.

[Elijahg]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.

Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.

Then if you don't have it, get Homebrew (at http://brew.sh/), and do

brew install --build-from-source python

This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.

You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.

To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py

Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3

Credit to Loumikas below for the last part!

Hope that helps someone.

[Elijahg]

I've put up a PR to address this in pyserial.

It has already been fixed in Python's repo (by Apple themselves) but it seems Apple are shipping an old version pf Python in Big Sur without the fix...

[Loumiakas]

I got it to work by installing pre-release version of pyserial with pip:
pip3 install pyserial --pre

[hawtdawg]

I got it to work by installing pre-release version of pyserial with pip:
pip3 install pyserial --pre

I just did that... and I also did brew install --build-from-source python. Both installations went through and were successful... and yet when I compile my code in Arduino I still get this:
Traceback (most recent call last): File "/Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 38, in <module> import serial ImportError: No module named serial exit status 1 Error compiling for board ESP32 Dev Module.

What am I doing wrong?

[Loumiakas]

I got it to work by installing pre-release version of pyserial with pip:
pip3 install pyserial --pre

I just did that... and I also did brew install --build-from-source python. Both installations went through and were successful... and yet when I compile my code in Arduino I still get this:
Traceback (most recent call last): File "/Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 38, in <module> import serial ImportError: No module named serial exit status 1 Error compiling for board ESP32 Dev Module.

What am I doing wrong?

I could be wrong, but it might be situation where epstool.py (in your case located /Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py ) is using a different python interpreter, which has an older version of pyserial.

If you open epstool.py and check first line, it should tell you which interpreter is being used. For example: #!/usr/local/Cellar/esptool/3.0/libexec/bin/python.

Then you could try directly invoking that python interpreter and seeing if following works: /usr/local/Cellar/esptool/3.0/libexec/bin/python -m serial.tools.list_ports -v.

If you get an error, then that means that particular interpreter does not have the pre-release version of pyserial installed. If that happens to be the case, you will need to install it e.g./usr/local/Cellar/esptool/3.0/libexec/bin/python -m pip install pyserial --pre

NOTE: I am using /usr/local/Cellar/esptool/3.0/libexec/bin/python as example. Interpreter being used by epstool.py could be different on your system.

[hawtdawg]

I got it to work by installing pre-release version of pyserial with pip:
pip3 install pyserial --pre

I just did that... and I also did brew install --build-from-source python. Both installations went through and were successful... and yet when I compile my code in Arduino I still get this:
Traceback (most recent call last): File "/Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 38, in <module> import serial ImportError: No module named serial exit status 1 Error compiling for board ESP32 Dev Module.
What am I doing wrong?

I could be wrong, but it might be situation where epstool.py (in your case located /Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py ) is using a different python interpreter, which has an older version of pyserial.

If you open epstool.py and check first line, it should tell you which interpreter is being used. For example: #!/usr/local/Cellar/esptool/3.0/libexec/bin/python.

Then you could try directly invoking that python interpreter and seeing if following works: /usr/local/Cellar/esptool/3.0/libexec/bin/python -m serial.tools.list_ports -v.

If you get an error, then that means that particular interpreter does not have the pre-release version of pyserial installed. If that happens to be the case, you will need to install it e.g./usr/local/Cellar/esptool/3.0/libexec/bin/python -m pip install pyserial --pre

Right! My first line in esptool.py is this: #!/usr/bin/env python

So then I tried what you said and got this:

Mate@Mates-MBP16 ~ % /usr/bin/env python -m serial.tools.list_ports -v
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named serial.tools
Mate@Mates-MBP16 ~ % 

I guess this is an error... now I'm trying to figure out what exactly I need to type in to get this to work but my path is quite different to yours, so I tried this but I'm obviously doing something wrong:

Mate@Mates-MBP16 ~ % /usr/bin/env python -m pip install pyserial --pre /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named pip

I'm sorry I'm not a programmer by any definition, any chance you could just write down something I could copy paste into Terminal?

[hawtdawg]

I got it to work by installing pre-release version of pyserial with pip:
pip3 install pyserial --pre

I just did that... and I also did brew install --build-from-source python. Both installations went through and were successful... and yet when I compile my code in Arduino I still get this:
Traceback (most recent call last): File "/Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 38, in <module> import serial ImportError: No module named serial exit status 1 Error compiling for board ESP32 Dev Module.
What am I doing wrong?

I could be wrong, but it might be the case, where epstool.py (in your case located /Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py ) is using a different python interpreter, which has an older version of pyserial. If you open epstool.py and check first line, it should tell you which interpreter is being used. For example: #!/usr/local/Cellar/esptool/3.0/libexec/bin/python. Then you could try directly invoking that python interpreter and seeing if following works: /usr/local/Cellar/esptool/3.0/libexec/bin/python -m serial.tools.list_ports -v. If you get an error, then that means that particular interpreter does not have the pre-release version of pyserial installed. If that is the case, you will need to install it e.g./usr/local/Cellar/esptool/3.0/libexec/bin/python -m pip install pyserial --pre

@Loumiakas is correct, Arduino is using the wrong version of Python - not the new one you just installed. If you have used brew install --build-from-source python you should change the shebang (line 1 in a script starting with !#) to !#/usr/local/bin/python3

This worked!! Thank you so much. I tried a variation of this before but I must have used the wrong path. Wow I thought this was never going to work.

[Elijahg]

I got it to work by installing pre-release version of pyserial with pip:
pip3 install pyserial --pre

I just did that... and I also did brew install --build-from-source python. Both installations went through and were successful... and yet when I compile my code in Arduino I still get this:
Traceback (most recent call last): File "/Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 38, in <module> import serial ImportError: No module named serial exit status 1 Error compiling for board ESP32 Dev Module.
What am I doing wrong?

I could be wrong, but it might be the case, where epstool.py (in your case located /Users/Username/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py ) is using a different python interpreter, which has an older version of pyserial. If you open epstool.py and check first line, it should tell you which interpreter is being used. For example: #!/usr/local/Cellar/esptool/3.0/libexec/bin/python. Then you could try directly invoking that python interpreter and seeing if following works: /usr/local/Cellar/esptool/3.0/libexec/bin/python -m serial.tools.list_ports -v. If you get an error, then that means that particular interpreter does not have the pre-release version of pyserial installed. If that is the case, you will need to install it e.g./usr/local/Cellar/esptool/3.0/libexec/bin/python -m pip install pyserial --pre

@Loumiakas is correct, Arduino is using the wrong version of Python - not the new one you just installed. If you have used brew install --build-from-source python you should change the shebang (line 1 in a script starting with !#) to !#/usr/local/bin/python3

This worked!! Thank you so much. I tried a variation of this before but I must have used the wrong path. Wow I thought this was never going to work.

No probs, glad it worked! :) I'll edit my original comment to clarify for others.

[wifwucite]

I think you need pip3 install pyserial --pre --force. Without the force option the existing version is not replaced.

[zsquareplusc]

should be fixed by v3.5

[xiyima]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.

Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.

Then if you don't have it, get Homebrew (at http://brew.sh/), and do

brew install --build-from-source python

This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.

You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.

To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py

Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3

Credit to Loumikas below for the last part!

Hope that helps someone.

Heyyy,
I tried to follow til renaming esptool.py into esptool, it didn't work for me in during compiling. it says for esptool tha "no such file or directory"

fork/exec /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool: no such file or directory
Error compiling for board DOIT ESP32 DEVKIT V1.

[xiyima]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.
Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.
Then if you don't have it, get Homebrew (at http://brew.sh/), and do
brew install --build-from-source python
This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.
You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.
To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py
Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool
This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3
Credit to Loumikas below for the last part!
Hope that helps someone.

Heyyy,
I tried to follow til renaming esptool.py into esptool, it didn't work for me in during compiling. it says for esptool tha "no such file or directory"

fork/exec /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool: no such file or directory
Error compiling for board DOIT ESP32 DEVKIT V1.

woo, found it was problem with extension. However i get a new error saying that permission denied ;((

[Elijahg]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.
Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.
Then if you don't have it, get Homebrew (at http://brew.sh/), and do
brew install --build-from-source python
This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.
You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.
To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py
Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool
This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3
Credit to Loumikas below for the last part!
Hope that helps someone.

Heyyy,
I tried to follow til renaming esptool.py into esptool, it didn't work for me in during compiling. it says for esptool tha "no such file or directory"
fork/exec /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool: no such file or directory
Error compiling for board DOIT ESP32 DEVKIT V1.

woo, found it was problem with extension. However i get a new error saying that permission denied ;((

You probably need to use chmod to add the execute bit to the esptool file:

chmod +x /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

[xiyima]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.
Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.
Then if you don't have it, get Homebrew (at http://brew.sh/), and do
brew install --build-from-source python
This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.
You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.
To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py
Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool
This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3
Credit to Loumikas below for the last part!
Hope that helps someone.

Heyyy,
I tried to follow til renaming esptool.py into esptool, it didn't work for me in during compiling. it says for esptool tha "no such file or directory"
fork/exec /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool: no such file or directory
Error compiling for board DOIT ESP32 DEVKIT V1.

woo, found it was problem with extension. However i get a new error saying that permission denied ;((

You probably need to use chmod to add the execute bit to the esptool file:

chmod +x /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

thanks for the fast reply! i forgot to add that...
i just did that to add the execute bit, but then a new error saying it can not use the usb port

Traceback (most recent call last):
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 3692, in
_main()
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 3685, in _main
main()
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 3339, in main
esp = chip_class(each_port, initial_baud, args.trace)
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 270, in init
self._port = serial.serial_for_url(port)
File "/usr/local/lib/python3.7/site-packages/serial/init.py", line 90, in serial_for_url
instance.open()
File "/usr/local/lib/python3.7/site-packages/serial/serialposix.py", line 325, in open
raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 16] could not open port /dev/cu.usbserial-0001: [Errno 16] Resource busy: '/dev/cu.usbserial-0001'
the selected serial port serial.serialutil.SerialException: [Errno 16] could not open port /dev/cu.usbserial-0001: [Errno 16] Resource busy: '/dev/cu.usbserial-0001'
does not exist or your board is not connected

[xiyima]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.
Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.
Then if you don't have it, get Homebrew (at http://brew.sh/), and do
brew install --build-from-source python
This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.
You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.
To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py
Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool
This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3
Credit to Loumikas below for the last part!
Hope that helps someone.

Heyyy,
I tried to follow til renaming esptool.py into esptool, it didn't work for me in during compiling. it says for esptool tha "no such file or directory"
fork/exec /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool: no such file or directory
Error compiling for board DOIT ESP32 DEVKIT V1.

woo, found it was problem with extension. However i get a new error saying that permission denied ;((

You probably need to use chmod to add the execute bit to the esptool file:
chmod +x /Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

thanks for the fast reply! i forgot to add that...
i just did that to add the execute bit, but then a new error saying it can not use the usb port

Traceback (most recent call last):
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 3692, in
_main()
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 3685, in _main
main()
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 3339, in main
esp = chip_class(each_port, initial_baud, args.trace)
File "/Users/Mxy/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool", line 270, in init
self._port = serial.serial_for_url(port)
File "/usr/local/lib/python3.7/site-packages/serial/init.py", line 90, in serial_for_url
instance.open()
File "/usr/local/lib/python3.7/site-packages/serial/serialposix.py", line 325, in open
raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 16] could not open port /dev/cu.usbserial-0001: [Errno 16] Resource busy: '/dev/cu.usbserial-0001'
the selected serial port serial.serialutil.SerialException: [Errno 16] could not open port /dev/cu.usbserial-0001: [Errno 16] Resource busy: '/dev/cu.usbserial-0001'
does not exist or your board is not connected

i changed to the USBtoUART port and it worked!!

[Elijahg]

Good to hear you fixed it! Looks like you probably had a terminal window open with screen or something similar using the port already. Maybe Arduino’s serial monitor, though that usually deactivates upon upload

[kolalde]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.

Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.

Then if you don't have it, get Homebrew (at http://brew.sh/), and do

brew install --build-from-source python

This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.

You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.

To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py

Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3

Credit to Loumikas below for the last part!

Hope that helps someone.

This worked for me!

My esptool location was a little different, I'm using 'portable', and I didn't need to enable the execute bit, it was already on. Thanks for the instructions.

[edmundsj]

This appears to be fixed in version 3.5 of pyserial. I did a
pip install pyserial --upgrade
and my issues with this problem were fixed.

[jimthedj65]

A better fix for this is to install a new version of Python with brew, which has been patched, this fixes it when using esptool.py directly, but esptool through Arduino also requires replacing the binary esptool in the Arduino ESP32 package with the source script.

Since I upgraded rather than doing a fresh install, I had another bug where brew failed because the Command Line Tools was too old, but software update thought it was current. So had to delete the old CommandLineTools in /Library/Developer/CommandLineTools. I also reinstalled Xcode to be safe. After deleting the CommandLineTools directory, do sudo xcode-select --install to install the new version, and afterward sudo xcodebuild -license to accept the new license.

Then if you don't have it, get Homebrew (at http://brew.sh/), and do

brew install --build-from-source python

This should then install a non-broken Python, you'll probably have to add /usr/local/bin/python to your $PATH to use it.

You may need to install pyserial with /usr/local/bin/pip3 install pyserial or /usr/local/bin/pip install pyserial depending on the Python version being used. Installing both won't do any harm.

To fix Arduino, you'll need to download the esptool.py source here: https://github.com/espressif/esptool/blob/master/esptool.py

Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

This might be enough to fix it. If you still get the same symbol not found error then open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool and change the first line from #!/usr/bin/env python to #!/usr/local/bin/python3

Credit to Loumikas below for the last part!

Hope that helps someone.

Thanks for posting this, this got me working on Big Sur after pulling my hair out. I followed all the instructions but skipped this step

Open ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ and rename esptool to esptool.old, and put the downloaded esptool.py into this same directory. When you've done that, rename esptool.py to esptool, and in the terminal set the executable bit: chmod +x ~/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool

I can now flash and update my firmware.

[sarkrui]

As an M1 Big Sur user, I found the similar issue when I ran esptool.py

Traceback (most recent call last):
[......]
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found

However, when I navigated to

/Users/p.xing/Library/Arduino15/packages/esp32/tools/esptool_py 

I found I actually have v3.0.0 esptool (Arduino IDE v1.8.13) installed instead of the 2.6.1 one. So, just incase you have the same situation. You can follow the below instructions to fix the issue.

1. Open and edit ~/.zshrc
2. Append the below

   export PATH="$PATH:/Users/p.xing/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0"

3. Save and reboot Terminal to reload the config.
4. Type esptool
5. You should be able to see the usage prompt!

[JuliusFue]

As an M1 Big Sur user, I found the similar issue when I ran esptool.py

Traceback (most recent call last):
[......]
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found

However, when I navigated to

/Users/p.xing/Library/Arduino15/packages/esp32/tools/esptool_py 

I found I actually have v3.0.0 esptool (Arduino IDE v1.8.13) installed instead of the 2.6.1 one. So, just incase you have the same situation. You can follow the below instructions to fix the issue.

1. Open and edit ~/.zshrc
2. Append the below

   export PATH="$PATH:/Users/p.xing/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0"

3. Save and reboot Terminal to reload the config.
4. Type esptool
5. You should be able to see the usage prompt!

hello, I am getting in line with this issue...
I followed all the hints and also edited the ~/.zshrc and I always get it working for some time, but when I want to use my ESP after some time again the Arduino IDE is not showing me the SerialPort to connect it, although my Mac is lists the connected device.

When I try to upload code anyways I get this error message:

esptool.py v3.2-dev
Traceback (most recent call last):
Serial port /dev/cu.usbserial-AI06B96J
File "/usr/local/lib/python3.9/site-packages/serial/serialposix.py", line 322, in open
self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
FileNotFoundError: [Errno 2] No such file or directory: '/dev/cu.usbserial-AI06B96J'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/julius/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool", line 4830, in
_main()
File "/Users/julius/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool", line 4823, in _main
main()
File "/Users/julius/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool", line 4281, in main
esp = esp or get_default_connected_device(ser_list, port=args.port, connect_attempts=args.connect_attempts,
File "/Users/julius/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool", line 121, in get_default_connected_device
_esp = chip_class(each_port, initial_baud, trace)
File "/Users/julius/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool", line 323, in init
self._port = serial.serial_for_url(port)
File "/usr/local/lib/python3.9/site-packages/serial/init.py", line 90, in serial_for_url
instance.open()
File "/usr/local/lib/python3.9/site-packages/serial/serialposix.py", line 325, in open
raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 2] could not open port /dev/cu.usbserial-AI06B96J: [Errno 2] No such file or directory: '/dev/cu.usbserial-AI06B96J'
the selected serial port serial.serialutil.SerialException: [Errno 2] could not open port /dev/cu.usbserial-AI06B96J: [Errno 2] No such file or directory: '/dev/cu.usbserial-AI06B96J'
does not exist or your board is not connected

I am working on BigSur 11.4 , Arduino 1.8.15 and using the AI-Thinker ESP32 Cam connect through an FTDI Board.

Does anyone have an idea what I did wrong?
If nothing helps would it be an option to install an older macOS version?

thanks in advance

[sarkrui]

But I can't find the file " ~/.zshrc " . Can you tell me where to look for it? What kind of file is it and how can I find it?

A file initialised with a dot is usually hidden in Finder by default in macOS, which is why you can't find it. So to modify it, you could use vim ~/.zshrc and append the esptool directory manually. Otherwise, you could also try the below command.

echo "#esptool v3.0 \nexport PATH="\$PATH:/Users/$USER/Library/Arduino15/packages/esp32/tools/esptool_py/3.0.0"" >> ~/.zshrc

[JuliusFue]

Got it working with a different board, so it must have something to do with the ftdi

[homonto]

I have solved the issue for Arduino on MacOS with the error related to no serial by below steps:

1. download pyserial pyserial-3.5.tar.gz
2. tar xfvz pyserial-3.5.tar.gz
3. cd pyserial-3.5
4. sudo /Applications/Xcode.app/Contents/Developer/usr/bin/python3 setup.py install

then Arduino started working as the issue was that python delivered with Xcode had no serial module installed

[hallard]

as indicated @homonto done same thing Monterey on mac M1 just python3 was not in same folder, changed last line all then went fine

sudo /Library/Developer/CommandLineTools/usr/bin/python3 setup.py install