Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing python on MacBook Pro M2 - ensurepip "died with <Signals.SIGABRT: 6>." #2707

Closed
13 tasks done
orlevii opened this issue Jun 1, 2023 · 26 comments
Closed
13 tasks done

Comments

@orlevii
Copy link

orlevii commented Jun 1, 2023

Too many issues will kill our team's development velocity, drastically.
Make sure you have checked all steps below.

Prerequisite

  • Make sure your problem is not listed in the common build problems.
  • Make sure no duplicated issue has already been reported in the pyenv issues. You should look for closed issues, too.
  • Make sure you are not asking us to help solving your specific issue.
    • GitHub issues is opened mainly for development purposes. If you want to ask someone to help solving your problem, go to some community site like Gitter, StackOverflow, etc.
  • Make sure your problem is not derived from packaging (e.g. Homebrew).
    • Please refer to the package documentation for the installation issues, etc.
  • Make sure your problem is not derived from plugins.
    • This repository is maintaining pyenv and the default python-build plugin only. Please refrain from reporting issues of other plugins here.

Description

  • Platform information (e.g. Ubuntu Linux 16.04): MacOS Ventura 13.4
  • OS architecture (e.g. amd64): arm
  • pyenv version: 2.3.18
  • Python version: 3.10.11
  • C Compiler information (e.g. gcc 7.3): Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  • Please attach the debug trace of the failing command as a gist:
    • Run env PYENV_DEBUG=1 <faulty command> 2>&1 | tee trace.log and attach trace.log. E.g. if you have a problem with installing Python, run env PYENV_DEBUG=1 pyenv install -v <version> 2>&1 | tee trace.log (note the -v option to pyenv install).
  • If you have a problem with installing Python, please also attach config.log from the build directory
    • The build directory is reported after the "BUILD FAILED" message and is usually under /tmp.
  • If the build succeeds but the problem is still with the build process (e.g. the resulting Python is missing a feature), please attach
    • the debug trace from reinstalling the faulty version with env PYENV_DEBUG=1 pyenv install -f -k -v <version> 2>&1 | tee trace.log
    • config.log from the build directory. When using pyenv install with -k as per above, the build directory will be under $PYENV_ROOT/sources.

I'm trying to install pytnon 3.10.11

python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.11.tar.xz...
-> https://www.python.org/ftp/python/3.10.11/Python-3.10.11.tar.xz
Installing Python-3.10.11...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 13.0 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631
Results logged to /var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631.log

Last 10 log lines:
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631/Python-3.10.11/Lib/ensurepip/__init__.py", line 287, in _main
    return _bootstrap(
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631/Python-3.10.11/Lib/ensurepip/__init__.py", line 203, in _bootstrap
    return _run_pip([*args, *_PACKAGE_NAMES], additional_paths)
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631/Python-3.10.11/Lib/ensurepip/__init__.py", line 104, in _run_pip
    return subprocess.run(cmd, check=True).returncode
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631/Python-3.10.11/Lib/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230601092224.55631/Python-3.10.11/python.exe', '-W', 'ignore::DeprecationWarning', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/tmp35tn17i5/setuptools-65.5.0-py3-none-any.whl\', \'/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/tmp35tn17i5/pip-23.0.1-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/tmp35tn17i5\', \'--root\', \'/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' died with <Signals.SIGABRT: 6>.
make: *** [install] Error 1

I've previously installed python 3.7.16 3.8.16 and 3.9.16 with no problem.
When I try to reinstall one of those versions, I get the same error.
So something was broken, I just can't tell what...

I've seen this issue - #2705 - I'm running on my ARM based brew, so it was not helpful.
(Also checked the trace.log to see if there's anything related to that, couldn't find anything)


Things I tried to do:

  • brew update & brew upgrade - Didn't change the outcome
  • Reinstalling xcode cli tools (xcode-select --install) - Didn't change the outcome
  • Installing python using Rosetta - works, but that's not what I'm looking for.

Logs

trace.log
config.log

@native-api
Copy link
Member

Please set the envvars that enable dyld diagnostic logging as per Logging Dynamic Loader Events -- Apple Documentation Archive and see if there'll be any details about that "missing symbol called" error.
We need to know which symbol failed to bind and in which library.

@orlevii
Copy link
Author

orlevii commented Jun 4, 2023

@native-api Thank you for responding.

So, I was running the installation like that (LMK if something is wrong):

export PYENV_DEBUG=1
export DYLD_PRINT_LIBRARIES=1
export DYLD_PRINT_LIBRARIES_POST_LAUNCH=1
export DYLD_PRINT_APIS=1
export DYLD_PRINT_STATISTICS=1
export DYLD_PRINT_INITIALIZERS=1
export DYLD_PRINT_SEGMENTS=1
export DYLD_PRINT_BINDINGS=1

pyenv install -v 3.10.11 2>&1 | tee trace.log

I couldn't find more information about the dyld[27040]: missing symbol called error

Attaching the new trace.log file:
trace.log

@native-api
Copy link
Member

native-api commented Jun 10, 2023

https://stackoverflow.com/questions/39623829/dyld-print-libraries-not-working says that the reason is SIP. It strips all DYLD_* envvars from any restricted process -- which includes shells that run Pyenv and build operations.

According to https://developer.apple.com/documentation/security/disabling_and_enabling_system_integrity_protection, it can only be temporarily disabled via Recovery mode.

@orlevii
Copy link
Author

orlevii commented Jun 11, 2023

@native-api
Thanks!

I've managed to disable SIP and rerun the installation.
The trace.log had 6.2 million rows, so I'm sharing only the relevant logs from the tail.

I found the missing symbol called error,
Looks like it's related to PyInit__csv

dyld[23001]: fixup: *0x0001044201B8 = 0x00018DA4E5E0 <_csv.cpython-310-darwin.so/bind#64>
dyld[23001]: mprotect 0x00010441C000->0x000104420000 to read-only
dyld[23001]:       dlopen(_csv.cpython-310-darwin.so) => 0x8faa1820
dyld[23001]: dlsym(0x8faa1820, "PyInit__csv")
dyld[23001]:      dlsym("PyInit__csv") => 0x104417460
dyld[23001]: missing symbol called
Traceback (most recent call last):
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/ensurepip/__init__.py", line 287, in _main
    return _bootstrap(
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/ensurepip/__init__.py", line 203, in _bootstrap
    return _run_pip([*args, *_PACKAGE_NAMES], additional_paths)
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/ensurepip/__init__.py", line 104, in _run_pip
    return subprocess.run(cmd, check=True).returncode
  File "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/Lib/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/python.exe', '-W', 'ignore::DeprecationWarning', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/tmpda5pyjuo/setuptools-65.5.0-py3-none-any.whl\', \'/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/tmpda5pyjuo/pip-23.0.1-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/tmpda5pyjuo\', \'--root\', \'/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' died with <Signals.SIGABRT: 6>.
make: *** [install] Error 1

I found this stackoverflow with similar issue: https://stackoverflow.com/questions/73583024/how-can-i-find-out-which-symbol-is-causing-dyld-missing-symbol-called that wasn't helpful for solving the problem.
I couldn't find more information online.

Attaching the log tail:
trace_tail.log

@native-api
Copy link
Member

native-api commented Jun 11, 2023

dyld[23001]:       dlopen(_csv.cpython-310-darwin.so) => 0x8faa1820
dyld[23001]: dlsym(0x8faa1820, "PyInit__csv")
dyld[23001]:      dlsym("PyInit__csv") => 0x104417460
dyld[23001]: missing symbol called

Okay. Let's see what's wrong with this symbol.

Please show the output of nm -g <path to _csv.cpython-310-darwin.so>.

For me, it shows:

<...>
00000000000038d0 T _PyInit__csv
<...>

full output

(The leading underscore is the result of compiler's mangling to indicate the C calling convention.)

@orlevii
Copy link
Author

orlevii commented Jun 11, 2023

Running:

nm -g "/private/var/folders/g7/g62ctbqd7r9_skkcy6122cp80000gq/T/python-build.20230611111718.7801/Python-3.10.11/build/lib.macosx-13.4-arm64-3.10/_csv.cpython-310-darwin.so"

Output:

                 U _PyArg_ParseTupleAndKeywords
                 U _PyArg_UnpackTuple
                 U _PyCallable_Check
                 U _PyDict_DelItem
                 U _PyDict_GetItemWithError
                 U _PyDict_Keys
                 U _PyDict_New
                 U _PyDict_SetItem
                 U _PyErr_Clear
                 U _PyErr_ExceptionMatches
                 U _PyErr_Format
                 U _PyErr_NoMemory
                 U _PyErr_Occurred
                 U _PyErr_SetString
                 U _PyExc_Exception
                 U _PyExc_KeyError
                 U _PyExc_SystemError
                 U _PyExc_TypeError
0000000000003460 T _PyInit__csv
                 U _PyIter_Next
                 U _PyList_Append
                 U _PyList_New
                 U _PyLong_AsLong
                 U _PyLong_FromLong
                 U _PyLong_Type
                 U _PyMem_Free
                 U _PyMem_Realloc
                 U _PyModuleDef_Init
                 U _PyModule_AddIntConstant
                 U _PyModule_AddObjectRef
                 U _PyModule_AddStringConstant
                 U _PyModule_AddType
                 U _PyModule_GetState
                 U _PyNumber_Check
                 U _PyNumber_Float
                 U _PyObject_GC_Del
                 U _PyObject_GC_Track
                 U _PyObject_GC_UnTrack
                 U _PyObject_GetAttrString
                 U _PyObject_GetIter
                 U _PyObject_IsTrue
                 U _PyObject_SelfIter
                 U _PyObject_Str
                 U _PyObject_VectorcallDict
                 U _PyThreadState_Get
                 U _PyTuple_Pack
                 U _PyType_FromModuleAndSpec
                 U _PyType_IsSubtype
                 U _PyUnicode_DecodeASCII
                 U _PyUnicode_FindChar
                 U _PyUnicode_FromKindAndData
                 U _PyUnicode_FromOrdinal
                 U _PyUnicode_GetLength
                 U __PyLong_AsInt
                 U __PyObject_GC_New
                 U __PyObject_LookupAttrId
                 U __PyObject_MakeTpCall
                 U __PyType_GetModuleByDef
                 U __PyType_Name
                 U __PyUnicode_Ready
                 U __Py_CheckFunctionResult
                 U __Py_Dealloc
                 U __Py_NoneStruct
                 U ___stack_chk_fail
                 U ___stack_chk_guard
                 U dyld_stub_binder

@zachmoshe
Copy link

Joining this thread. I'm seeing the same error when building Python 3.10 (it gets the latest which is 3.10.12). My setup is:

  • Platform information (e.g. Ubuntu Linux 16.04): MacOS Ventura 13.4
  • OS architecture (e.g. amd64): Intel Core i9
  • pyenv version: 2.3.19
  • Python version: 3.10.12
  • C Compiler information (e.g. gcc 7.3): Not sure what to paste here, this is my clang --version:
> clang --version
Homebrew clang version 16.0.5
Target: x86_64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

Any idea why the error refers to an error when running python.exe where we're clearly compiling for MacOS?

@native-api
Copy link
Member

native-api commented Jun 16, 2023

@anton-petrov @joshfriend Could you take a look? I cannot reproduce the problem -- maybe you will be able to?
The except from #2707 (comment) doesn't seem to be the cause -- the binding appears to succeed despite the name being different. So I have no leads.

@orlevii @zachmoshe could you try reproducing the problem while having enabled ALL DYLD diagnostic envvars mentioned on your dyld manpage?
The messages in @orlevii 's log seem to be a red herring as per above -- so we still don't know which function fails to bind.
no use, see below

@native-api
Copy link
Member

native-api commented Jun 16, 2023

Neither can I find the source code that produces that "missing symbol called" at https://opensource.apple.com -- to find out what we can set to get more information about it.

Searching with grep, I do see that the message is in /usr/lib/dyld.

@native-api
Copy link
Member

native-api commented Jun 16, 2023

Okay... it's STRAIGHT-OUT IMPOSSIBLE to get any more info from MacOS:

void _dyld_missing_symbol_abort()
{
    // We don't know the name of the lazy symbol that is missing.
    // dyld3 binds all such missing symbols to this one handler.
    // We need the crash log to contain the backtrace so someone can
    // figure out the symbol.

    auto allImageInfos = gAllImages.oldAllImageInfo();
    allImageInfos->errorKind           = DYLD_EXIT_REASON_SYMBOL_MISSING;
    allImageInfos->errorClientOfDylibPath   = "<unknown>";
    allImageInfos->errorTargetDylibPath     = "<unknown>";
    allImageInfos->errorSymbol              = "<unknown>";

    halt("missing lazy symbol called");
}

"MacOS... bloody useless!"

@native-api
Copy link
Member

native-api commented Jun 16, 2023

So one has to run the build under a debugger (including child processes) and set a breakpoint to this function.

  • Not sure if one can make the debugger use any "corresponding source code" for dyld from Apple, so maybe need to disassemble /usr/lib/dyld to get the function's address (first, find out the string's address, then see which code references it).

Once the breakpoin is hit, see up the call stack where it has been called from.

@native-api
Copy link
Member

native-api commented Jun 16, 2023

The source code references making a "crash log" that might have the backtrace we need.

@native-api
Copy link
Member

native-api commented Jun 16, 2023

According to https://www.stata.com/support/faqs/mac/finding-mac-crash-log/, crash logs are placed into ~/Library/Logs/DiagnosticReports/ (which is hidden in Finder, the link has instructions on how to see it).

@native-api
Copy link
Member

Any idea why the error refers to an error when running python.exe where we're clearly compiling for MacOS?

"python.exe" is an intermediate name used by CPython's build in MacOS for some reason.

@native-api
Copy link
Member

@orlevii @zachmoshe In any case, googling for the "missing symbol called", as a workaround, they suggest reinstalling Homebrew and rebuilding everything that uses it.
There's definitely an incompatibility between some of the installed library versions involved, we just don't know which exactly.

@anton-petrov
Copy link
Member

anton-petrov commented Jun 17, 2023

Just tested on my laptop with m1 pro and macOS 13.4, everything is ok!

❯ brew install pyenv
❯ pyenv install 3.10.12
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.12.tar.xz...
-> https://www.python.org/ftp/python/3.10.12/Python-3.10.12.tar.xz
Installing Python-3.10.12...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.12 to /Users/anton/.pyenv/versions/3.10.12
❯ pyenv --version
pyenv 2.3.20
❯ which pyenv
/opt/homebrew/bin/pyenv

@wesyoung
Copy link

i think the solution is removing brew and re-installing it + pyenv from scratch. i carried my old ~/.pyenv env from an x86 laptop (macos transfer) to my M2 max, everything "just worked" (didn't think about this) until i tried to install a new version of python via pyenv..

after doing that, was able to install 3.9.17, 3.10.12 and 3.11.4 no problem.

ymmv

(make sure you have the correct CPP/LDFLAGS, readline, openssl, etc.. in place too)

@orlevii
Copy link
Author

orlevii commented Jun 19, 2023

Update:
The problem has been resolved by uninstalling homebrew and performing a fresh reinstallation.

It should be noted that uninstalling homebrew will result in the removal of all installed formulaes.

I'm still unsure which library caused the problem in my case and whether there are more effective ways to handle this problem in the future.


Thanks @native-api for helping me out

@native-api
Copy link
Member

Another possible option would be to reinstall all the Homebrew formulae.

It's likely that Apple have bumped some library that some Homebrew packages were using -- so if you've migrated the Homebrew installation from an older MacOS version, they no longer match that dependency.

@vhnguyenae
Copy link

i think the solution is removing brew and re-installing it + pyenv from scratch. i carried my old ~/.pyenv env from an x86 laptop (macos transfer) to my M2 max, everything "just worked" (didn't think about this) until i tried to install a new version of python via pyenv..

after doing that, was able to install 3.9.17, 3.10.12 and 3.11.4 no problem.

ymmv

(make sure you have the correct CPP/LDFLAGS, readline, openssl, etc.. in place too)

Thanks, reinstall brew helped.
BUT I also need to do a computer restart, since I ran into the same error again even after purging and reinstall brew several times. I tried my luck by restarting the computer and voila 🎉

@lexmiln
Copy link

lexmiln commented Nov 8, 2023

I found this thread after seeing

subprocess.CalledProcessError: Command '['/Users/alex/Louise/wyoming-slack/.venv/bin/python3.11', '-m', 'ensurepip', '--upgrade', '--default-pip']' died with <Signals.SIGABRT: 6>.

while running a setup script on a MacBook Pro M1.

I resolved the issue with just brew install python3, which replaced my homebrew Python 3.11.5 with Python 3.11.6. The setup script then succeeded.

@netgitdom
Copy link

On my M1 MacBook Air, for which I had started with the x86 Rosetta Homebrew back in 2020, reinstalling peen fixed the

dyld[74020]: missing symbol called

error for me:

brew reinstall $(brew deps pyenv) pyenv

@martinfrouin
Copy link

On my M2, the solution was to run the command with arch -arm64 prefix

@siennamw
Copy link

siennamw commented Jan 3, 2024

My old Homebrew installation was stubborn, so simply uninstalling and reinstalling it didn't solve the problem. I ended up needing to take the nuclear approach to fully remove the old Homebrew:

rm -rf /usr/local/bin/brew
sudo rm -rf /usr/local/Caskroom
sudo rm -rf /usr/local/Cellar
sudo rm -rf /usr/local/Frameworks
sudo rm -rf /usr/local/Homebrew

Once this was done, the uninstall/reinstall suggestions above seemed to do the trick.

If you're seeing this error on an M1 or M2, I suggest migrating your Homebrew installation (here's one guide) before trying anything else.

@ClementWalter
Copy link

ClementWalter commented Feb 27, 2024

I had this after a migration from intel to mac m2.
I realized that both pyenv and brew were in intel compatibility/rosetta mode.

Removing both the old pyenv and brew, then

brew install pyenv
pyenv install x.x.x.

worked as expected

@patnir411
Copy link

From this duplicate thread

arch -x86_64 pyenv install 3.x.x

That's what solved it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests