Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyenv install fails to build 2.7.14 with ImportError: No module named pyexpat #1066

Open
justinjdickow opened this issue Jan 1, 2018 · 53 comments

Comments

@justinjdickow
Copy link

commented Jan 1, 2018

pyenv version: 1.2.0
MacOS 10.13.2
python-build version: 20160602

I've run

brew install readline xz and xcode-select --install

I'm sourcing flags using the following

export CFLAGS="-I$(xcrun --show-sdk-path)/usr/include"
export CFLAGS="-I$(brew --prefix readline)/include $CFLAGS"
export LDFLAGS="-L$(brew --prefix readline)/lib $LDFLAGS"
export CFLAGS="-I$(brew --prefix openssl)/include $CFLAGS"
export LDFLAGS="-L$(brew --prefix openssl)/lib $LDFLAGS"
export PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig"

When I run pyenv install 2.7.14 I get the following error

Traceback (most recent call last):
  File "get-pip.py", line 20061, in <module>
    main()
  File "get-pip.py", line 194, in main
    bootstrap(tmpdir=tmpdir)
  File "get-pip.py", line 82, in bootstrap
    import pip
  File "/var/folders/t5/4dd3lpyj50j05f9xn49hzj8n8mjnmf/T/tmpXrOLYp/pip.zip/pip/__init__.py", line 26, in <module>
  File "/var/folders/t5/4dd3lpyj50j05f9xn49hzj8n8mjnmf/T/tmpXrOLYp/pip.zip/pip/utils/__init__.py", line 27, in <module>
  File "/var/folders/t5/4dd3lpyj50j05f9xn49hzj8n8mjnmf/T/tmpXrOLYp/pip.zip/pip/_vendor/pkg_resources/__init__.py", line 1027, in <module>
  File "/var/folders/t5/4dd3lpyj50j05f9xn49hzj8n8mjnmf/T/tmpXrOLYp/pip.zip/pip/_vendor/pkg_resources/__init__.py", line 1030, in Environment
  File "/var/folders/t5/4dd3lpyj50j05f9xn49hzj8n8mjnmf/T/tmpXrOLYp/pip.zip/pip/_vendor/pkg_resources/__init__.py", line 272, in get_supported_platform
  File "/var/folders/t5/4dd3lpyj50j05f9xn49hzj8n8mjnmf/T/tmpXrOLYp/pip.zip/pip/_vendor/pkg_resources/__init__.py", line 444, in _macosx_vers
  File "/usr/local/opt/pyenv/versions/2.7.14/lib/python2.7/platform.py", line 764, in mac_ver
    info = _mac_ver_xml()
  File "/usr/local/opt/pyenv/versions/2.7.14/lib/python2.7/platform.py", line 741, in _mac_ver_xml
    pl = plistlib.readPlist(fn)
  File "/usr/local/opt/pyenv/versions/2.7.14/lib/python2.7/plistlib.py", line 78, in readPlist
    rootObject = p.parse(pathOrFile)
  File "/usr/local/opt/pyenv/versions/2.7.14/lib/python2.7/plistlib.py", line 401, in parse
    from xml.parsers.expat import ParserCreate
  File "/usr/local/opt/pyenv/versions/2.7.14/lib/python2.7/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ImportError: No module named pyexpat
@webknjaz

This comment has been minimized.

Copy link

commented Jan 15, 2018

Hi, I'm getting exactly same issue, when trying to pyenv install 3.6.4 in CircleCI.

Here's the build log:
https://circleci.com/gh/cherrypy/cheroot/291

@webknjaz

This comment has been minimized.

Copy link

commented Jan 15, 2018

@yyuu @joshfriend any ideas regarding this failure?

@webknjaz

This comment has been minimized.

Copy link

commented Jan 16, 2018

@natecox

This comment has been minimized.

Copy link

commented Jan 16, 2018

Have you updated brew and installed the updated pyenv? The current pyenv version is 1.2.1, and it looks like you're running 1.2.0.

The newest version is working flawlessly for me on 10.13.2.

Edit: it looks like your CircleCI config is explicitly set to avoid updates, which may also be the problem there.

@webknjaz

This comment has been minimized.

Copy link

commented Jan 17, 2018

@natecox yeah, it takes some time to to upgrade, so I've disabled it, but this problem existed before that. I just wanted to speed-up testing.

Should I install pyenv via brew as well?

@justinjdickow

This comment has been minimized.

Copy link
Author

commented Jan 17, 2018

Same error with pyenv 1.2.1 and macOS 10.13.2

@webknjaz

This comment has been minimized.

Copy link

commented Jan 17, 2018

1.2.1 + 10.13.5 doesn't work either: https://circleci.com/gh/cherrypy/cheroot/298

@webknjaz

This comment has been minimized.

Copy link

commented Jan 17, 2018

So it basically fails to build pyexpat module:

clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/distiller/.pyenv/versions/3.6.4/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/distiller/.pyenv/versions/3.6.4/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/distiller/.pyenv/versions/3.6.4/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -I/Applications/Xcode-9.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/Users/distiller/.pyenv/versions/3.6.4/include build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/_decimal.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/basearith.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/constants.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/context.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/convolute.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/crt.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/difradix2.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/fnt.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/fourstep.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/io.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/memory.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/mpdecimal.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/numbertheory.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/sixstep.o build/temp.macosx-10.12-x86_64-3.6/private/var/folders/ms/xg67k5sn16xc7sdr_w3q45840000gn/T/python-build.20180115145538.1345/Python-3.6.4/Modules/_decimal/libmpdec/transpose.o -L/usr/local/opt/readline/lib -L/Users/distiller/.pyenv/versions/3.6.4/lib -L/usr/local/opt/openssl/lib -L/usr/local/lib -o build/lib.macosx-10.12-x86_64-3.6/_decimal.cpython-36m-darwin.so
*** WARNING: renaming "pyexpat" since importing it failed: dlopen(build/lib.macosx-10.12-x86_64-3.6/pyexpat.cpython-36m-darwin.so, 2): Symbol not found: _XML_ErrorString
  Referenced from: build/lib.macosx-10.12-x86_64-3.6/pyexpat.cpython-36m-darwin.so
  Expected in: flat namespace
 in build/lib.macosx-10.12-x86_64-3.6/pyexpat.cpython-36m-darwin.so
*** WARNING: renaming "_elementtree" since importing it failed: PyCapsule_Import could not import module "pyexpat"

Python build finished successfully!
The necessary bits to build these optional modules were not found:
ossaudiodev           spwd                                     
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
atexit                pwd                   time               

Failed to build these modules:
readline                                                       


Following modules built successfully but were removed because they could not be imported:
_elementtree          pyexpat                                  

and then pip run fails to import it.

but installing expat doesn't fix the problem..

@justinjdickow

This comment has been minimized.

Copy link
Author

commented Jan 17, 2018

@webknjaz It makes sense that installing expat doesn't fix the problem. pyenv is creating a whole new installation of python that is independent and it's trying to install pyexpat there.

@natecox

This comment has been minimized.

Copy link

commented Jan 17, 2018

The expat issue is the same I was having pre 1.2.1, but updating did fix the issue because they patched the builder. Aside from that, I'm not sure why it wouldn't be working

@webknjaz

This comment has been minimized.

Copy link

commented Jan 17, 2018

I don't see anything in log noticing installation of expat: AFAIU it just builds pyexpat module during the python build process. I thought it should be linked against some external libexpat

@justinjdickow

This comment has been minimized.

Copy link
Author

commented Jan 17, 2018

I guess maybe I don't understand it :P I could see that as well

@webknjaz

This comment has been minimized.

Copy link

commented Jan 17, 2018

It looks just like #1073, except for different dependency failing

@natecox

This comment has been minimized.

Copy link

commented Jan 17, 2018

I just went back and installed a new version to make sure that this was indeed still working.

•100% ➜ pyenv install 3.6.4
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.6.4.tar.xz...
-> https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz
Installing Python-3.6.4...
python-build: use readline from homebrew
Installed Python-3.6.4 to /Users/nate/.pyenv/versions/3.6.4

My brew info for pyenv

~
•100% ➜ brew info pyenv
pyenv: stable 1.2.1 (bottled), HEAD
Python version management
https://github.com/pyenv/pyenv
/usr/local/Cellar/pyenv/1.1.5 (594 files, 2.3MB)
  Poured from bottle on 2017-11-30 at 12:43:09
/usr/local/Cellar/pyenv/HEAD-6f27c91 (584 files, 2.3MB) *
  Built from source on 2018-01-13 at 11:39:57
/usr/local/Cellar/pyenv/HEAD-85a66a9 (579 files, 2.2MB)
  Built from source on 2017-12-04 at 13:34:03
/usr/local/Cellar/pyenv/HEAD-e71ac9e (580 files, 2.3MB)
  Built from source on 2017-12-14 at 15:55:34
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/pyenv.rb
==> Dependencies
Recommended: autoconf ✔, pkg-config ✔, openssl ✔, readline ✔
==> Options
--without-autoconf
	Build without autoconf support
--without-openssl
	Build without openssl support
--without-pkg-config
	Build without pkg-config support
--without-readline
	Build without readline support
--HEAD
	Install HEAD version
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh functions have been installed to:
  /usr/local/share/zsh/site-functions

Pyenv version

~
•100% ➜ pyenv --version
pyenv 1.2.1

Hopefully this helps shine a light on the issue for you.

@webknjaz

This comment has been minimized.

Copy link

commented Jan 18, 2018

Turns out that adding

export CFLAGS="-I$(xcrun --show-sdk-path)/usr/include $CFLAGS"

breaks the build.

@webknjaz

This comment has been minimized.

Copy link

commented Jan 18, 2018

It lead me to wrong guesses at the beginning when I combined installation of several versions asynchronously. And then adding CFLAGS caused build error with pyexpat.
After I removed CFLAGS I realised that most of builds are successful, except for pypy2.7-5.10.0, which is failing for different reason.

Here's what works for me (except pypy2.7-5.10.0):

# No additional env vars
brew install pyenv readline xz
pyenv install {{version}}

https://circleci.com/gh/cherrypy/cheroot/323

@jklemm

This comment has been minimized.

Copy link

commented May 7, 2018

Thanks @webknjaz!

I have this error on Mac OS High Sierra v 10.13.4 when trying to install python 2.7.12

Unset the variable saves the day!

unset CFLAGS
brew install pyenv readline xz
pyenv install {{version}}

robnagler added a commit to biviosoftware/home-env that referenced this issue Mar 4, 2019

Use $HOME so relocatable; protect not strict ops
added bivio_not_strict_cmd to protect operations that
might fail with undefined var or error.
Remove CFLAGS from pyenv build on Darwin because no longer necessary
see pyenv/pyenv#1066
@danwiesenthal

This comment has been minimized.

Copy link

commented Jun 13, 2019

I'm seeing this issue again with macOS 10.14.5 and pyenv 1.2.12. I was able to install python 3.6.7, but can't install anything on the 3.7.x branch, for lack of pyexpat. Here's the output:

pyenv install 3.7.3                                                                                                                               ▸▸▸▸▸▸▸▸▸▸
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.3.tar.xz...
-> https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
Installing Python-3.7.3...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 10.14.5 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190613152307.55015
Results logged to /var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190613152307.55015.log

Last 10 log lines:
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmphic5qgk5/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/cli/cmdoptions.py", line 22, in <module>
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmphic5qgk5/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/utils/hashes.py", line 10, in <module>
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmphic5qgk5/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/utils/misc.py", line 21, in <module>
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmphic5qgk5/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/pkg_resources/__init__.py", line 35, in <module>
  File "/private/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190613152307.55015/Python-3.7.3/Lib/plistlib.py", line 65, in <module>
    from xml.parsers.expat import ParserCreate
  File "/private/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190613152307.55015/Python-3.7.3/Lib/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ModuleNotFoundError: No module named 'pyexpat'
make: *** [install] Error 1
@iamlordaubrey

This comment has been minimized.

Copy link

commented Jun 14, 2019

@danwiesenthal I had this same issue just yesterday. Fixed it by downloading and reinstalling command line tools. Also downloaded and reinstalled Xcode itself (not sure this helped)
https://developer.apple.com/download/more/?=command%20line%20tools

@danwiesenthal

This comment has been minimized.

Copy link

commented Jun 14, 2019

Ah, alas. I tried to start fresh: I uninstalled Xcode and Command Line Tools (including deleting their directories, etc, per googling instructions on how to remove them). I uninstalled everything brew had installed. I reinstalled both Xcode and the Command Line tools. I brew installed openssl, and set the compiler flags to find openssl per brew's suggestion. I ran the command to link to the command line tool headers. I brew installed pyenv. And yet... back where I started.

pyenv install 3.7.2                                                                                                                                                     
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.2.tar.xz...
-> https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tar.xz
Installing Python-3.7.2...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 10.14.5 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190614151842.24131
Results logged to /var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190614151842.24131.log

Last 10 log lines:
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmpvv82ceeh/pip-18.1-py2.py3-none-any.whl/pip/_internal/cli/cmdoptions.py", line 20, in <module>
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmpvv82ceeh/pip-18.1-py2.py3-none-any.whl/pip/_internal/utils/hashes.py", line 10, in <module>
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmpvv82ceeh/pip-18.1-py2.py3-none-any.whl/pip/_internal/utils/misc.py", line 21, in <module>
  File "/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/tmpvv82ceeh/pip-18.1-py2.py3-none-any.whl/pip/_vendor/pkg_resources/__init__.py", line 35, in <module>
  File "/private/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190614151842.24131/Python-3.7.2/Lib/plistlib.py", line 65, in <module>
    from xml.parsers.expat import ParserCreate
  File "/private/var/folders/z3/g070g97n7qv88q0qjcldl9h00000gn/T/python-build.20190614151842.24131/Python-3.7.2/Lib/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ModuleNotFoundError: No module named 'pyexpat'
make: *** [install] Error 1
@lgw4

This comment has been minimized.

Copy link

commented Jun 15, 2019

@danwiesenthal I fixed this by installing the Command_Line_Tools_for_Xcode_11_Beta.dmg package from the site linked to by @iamlordaubrey above. You can verify which version you have installed by running pkgutil --pkg-info=com.apple.pkg.CLTools_Executables. You should see output similar to this:

package-id: com.apple.pkg.CLTools_Executables
version: 11.0.0.0.1.1559496560
volume: /
location: /
install-time: 1560561423
groups: com.apple.FindSystemFiles.pkg-group
@neverfox

This comment has been minimized.

Copy link

commented Jun 17, 2019

I reinstalled the tools which are still at v10.x and can now install 3.7.3:

❯ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 10.2.1.0.1.1554506761
volume: /
location: /
install-time: 1560789554
groups: com.apple.FindSystemFiles.pkg-group
@danwiesenthal

This comment has been minimized.

Copy link

commented Jun 17, 2019

THRILLED to say that I've got a working system now. Thank you all for your help! :)

For whatever reason, the previous stable build of CLT (for macOS 10.14, Xcode 10.2.1) did NOT work for me (even though I'm not running anything else beta; no Catalina, no Xcode 11; running vanilla macOS 10.14.5), regardless of compiler flags, header re-linking, etc.

What DID work for me was installing the Xcode CLT 11 Beta 1. With that, and these compiler flags:

  • $CFLAGS="-I/usr/local/Cellar/openssl/1.0.2s/include"
  • $LDFLAGS="-L/usr/local/Cellar/openssl/1.0.2s/lib"

I was successfully able to install pyenv and install arbitrary Python versions successfully without the pyexpat error. Hooray!

For the curious, here's the output from pkgutil for the working configuration (in conjunction with aforementioned flags):

pkgutil --pkg-info=com.apple.pkg.CLTools_Executables

package-id: com.apple.pkg.CLTools_Executables
version: 11.0.0.0.1.1559496560
volume: /
location: /
install-time: 1560795402
groups: com.apple.FindSystemFiles.pkg-group
@sophiebits-humu

This comment has been minimized.

Copy link

commented Jun 18, 2019

It appears that the zlib fix last week (4157973; cc @mikeroll @joshfriend) exacerbates this problem. In particular, it causes pyexpat to fail to link when installing on a fresh Mojave system with the 10.14 Command Line Tools and SDK headers (/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg) installed.

Here, a log from my coworker's machine when installing Python 3.7.3 using pyenv 1.2.12:

clang -bundle -undefined dynamic_lookup -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/.pyenv/versions/3.7.3/lib -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/.pyenv/versions/3.7.3/lib -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/.pyenv/versions/3.7.3/lib -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include -I/Users/brianh/usr/local/opt/readline/include -I/Users/brianh/usr/local/opt/readline/include -I/Users/brianh/.pyenv/versions/3.7.3/include build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/_ctypes.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/callbacks.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/callproc.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/stgdict.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/cfield.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/malloc_closure.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/darwin/dlfcn_simple.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/ffi.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/x86/darwin64.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/x86/x86-darwin.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/x86/x86-ffi_darwin.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/x86/x86-ffi64.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/powerpc/ppc-darwin.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/powerpc/ppc-darwin_closure.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/powerpc/ppc-ffi_darwin.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/lz/81t0dgy54kg4jmlyn21vz6bh0000gn/T/python-build.20190614142435.36559/Python-3.7.3/Modules/_ctypes/libffi_osx/powerpc/ppc64-darwin_closure.o -L/Users/brianh/usr/local/opt/readline/lib -L/Users/brianh/.pyenv/versions/3.7.3/lib -L/usr/local/lib -o build/lib.macosx-10.14-x86_64-3.7/_ctypes.cpython-37m-darwin.so
ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
*** WARNING: renaming "pyexpat" since importing it failed: dlopen(build/lib.macosx-10.14-x86_64-3.7/pyexpat.cpython-37m-darwin.so, 2): Symbol not found: _XML_ErrorString
  Referenced from: build/lib.macosx-10.14-x86_64-3.7/pyexpat.cpython-37m-darwin.so
  Expected in: flat namespace
 in build/lib.macosx-10.14-x86_64-3.7/pyexpat.cpython-37m-darwin.so
*** WARNING: renaming "_elementtree" since importing it failed: PyCapsule_Import could not import module "pyexpat"

Most people who have run into this problem in the past recommend unsetting CFLAGS in the global environment to work around the problem. However, the aforementioned commit means that the use_xcode_sdk_zlib branch is taken on any machine where xcrun --show-sdk-path returns successfully, causing CFLAGS to be set to include -I${xc_sdk_path}/usr/include. As far as I can tell, there is no way to override this behavior using command line flags or environment variables.

We opted to downgrade to pyenv 1.2.11 and were able to successfully build on that version (using a copy of zlib from Homebrew).

I think one possible fix would be to swap the order of the checks (that is, use_homebrew_zlib || use_xcode_sdk_zlib instead of use_xcode_sdk_zlib || use_homebrew_zlib). Then you would either need to (a) install zlib using brew, as was required before or (b) upgrade to the beta Command Line Tools (as mentioned above). Whereas right now the latter appears to be the only option using pyenv 1.2.12.

@cicorias

This comment has been minimized.

Copy link

commented Jun 19, 2019

same boat as @sophiebits-humu -- rolling / downgrading Pyenv back to 1.2.11 fixes this for me.

@amyccaldwell

This comment has been minimized.

Copy link

commented Jun 19, 2019

Like @neverfox, reinstalling the command line tools (still at 10.2.1.0.1.1554506761) resolved the pyexpat errors for me.

@phazel

This comment has been minimized.

Copy link

commented Jun 20, 2019

@iamlordaubrey Thankyou for figuring out the solution! I guess xcode-select --install still installs the old xcode command line tools because the new one for Mojave is still in beta.

@hrdchz

This comment has been minimized.

Copy link

commented Jun 20, 2019

I my case installing Command Line Tools for Xcode 10.2.1, using the dmg installer which makes you agree to the license, seems to have been the solution. I never use Xcode so I at least ran it once out of superstition.

Additionally I did brew reinstall pyenv readline xv after the CLI tools install and included and exports in my startup files (.zshrc / .bash_profile): brew info openssl / brew info readline

EDIT appears to be same solution as @amyccaldwell @neverfox etc

@henriquebastos

This comment has been minimized.

Copy link

commented Jun 23, 2019

I had the same problem with version 3.7.3.

The solution I found on the web is:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.7.3
@anilsg

This comment has been minimized.

Copy link

commented Jun 27, 2019

I've worked through almost all of the offered solutions comprehensively.
To potentially save others some time I summarise:

As per @cicorias and @sophiebits-humu I expect downgrading Pyenv back to 1.2.11 would fix (not tested). This may not be (easily) available through Homebrew if you don't have a previous local install.

As per @sophiebits-humu 8 days ago, upgrade to the beta Command Line Tools worked for me with pyenv 1.2.12. According to @sophiebits-humu this is the only available route with pyenv 1.2.12, and I believe her.

  • I did not try Command_Line_Tools_for_Xcode_11_Beta.dmg.
  • I downloaded Command_Line_Tools_for_Xcode_11_Beta_2.dmg from Apple Developer.
  • I installed Beta 2 CLTools manually using the DMG and agreeing to the license.
  • No Packages are provided in /Library/Developer/CommandLineTools/ for this version so there was no header package to install.
  • pkgutil --pkg-info=com.apple.pkg.CLTools_Executables reports version: 11.0.0.0.1.1560537986.
  • I then ran brew reinstall pyenv readline openssl.

Then pyenv install 2.7.16 and pyenv install 3.7.3 both succeeded.

As long as I had Command_Line_Tools_macOS_10.14_for_Xcode_10.2.1.dmg and pyenv 1.2.12 installed (pkgutil --pkg-info=com.apple.pkg.CLTools_Executables # version: 10.2.1.0.1.1554506761) the other options all failed.

As per @hrdchz @amyccaldwell @neverfox resolve from 6 days ago installing Command_Line_Tools_macOS_10.14_for_Xcode_10.2.1.dmg manually agreeing to license
and /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg manually agreeing to license pyenv install 3.7.3 still failed for me with ModuleNotFoundError: No module named 'pyexpat'

@hrdchz @amyccaldwell @neverfox route from 6 days ago also failed.

As per @amyccaldwell 7 days ago (apparently like @neverfox) reinstalling the same latest (not beta) command line tools still at 10.2.1.0.1.1554506761 even including the @hrdchz actlons to reinstall pyenv readline and all exports included still failed.

I didn't try @henriquebastos option using Xcode (SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.7.3) since @markosamuli commented that for him Python 3.7.3 install still fails on Travis macOS 10.14

@j9ac9k

This comment has been minimized.

Copy link

commented Jul 3, 2019

FWIW the following let me install python 3.8-dev on macOS mojave:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.8-dev
@tilgovi

This comment has been minimized.

Copy link

commented Jul 3, 2019

I was getting the pyexpat error. I fixed it by disabling SIP and manually uninstalling everything installed by macOS_SDK_headers_for_macOS_10.14.pkg. That's not fun and not easy and after telling a team of engineers to install that package for things to work before, I'm not going to try to tell them to uninstall it this way for things to work now.

Do we know why the beta command line tools works? Does it remove the old macOS_SDK_headers_for_macOS_10.14.pkg? If so, installing the beta, even if it's uninstalled and removed after, might be a great way to get around disabling SIP to uninstall the old headers.

@anilsg

This comment has been minimized.

Copy link

commented Jul 3, 2019

@tilgovi

This comment has been minimized.

Copy link

commented Jul 3, 2019

I'm speaking specifically of the extra, optional headers package that installs common includes into /usr/include, outside the SDK directory.

@b-gale

This comment has been minimized.

Copy link

commented Jul 4, 2019

@j9ac9k 's solution worked for me for 3.5.7. system: pyenv 1.2.12, macOS 10.14.5, CL Tools Executables 10.2.1.0.1.1554506761. This is IMO the most elegant solution as it's one line and installs/uninstalls required.

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.5.7

@anilsg

This comment has been minimized.

Copy link

commented Jul 4, 2019

@b-gale , @j9ac9k 's solution includes the full install of Xcode, as does your quote of the solution. My statement of the requirement for Beta CLTools was based on not installing Xcode itself, as stated, because others identified it did not work in their case, and I didn't want Xcode. It seems installing Xcode works for some (e.g. you and j9), but not others (e.g. @markosamuli).

So there remains at least the posssibility of solutions by rolling back to earlier pyenv, installing full XCode, but otherwise you will need beta CLTools.

@bosconb

This comment has been minimized.

Copy link

commented Jul 7, 2019

same issue,when macOS is 14.3

$ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 10.1.0.0.1.1539992718
volume: /
location: /
install-time: 1550213543
groups: com.apple.FindSystemFiles.pkg-group

after upgrade my macos to 10.14.6 Beta, pyenv success to install python 3.7.3

$ pyenv install 3.7.3
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.3.tar.xz...
-> https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
Installing Python-3.7.3...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.7.3 to /Users/dazkarieh/.pyenv/versions/3.7.3
@michaelkarrer81

This comment has been minimized.

Copy link

commented Jul 15, 2019

pyenv install 2.7.16 worked for me after installing command line tools beta 2
But i am still stuck and can not use python because i can not pip install readline !

Error message:

    clang -fno-strict-aliasing -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DHAVE_RL_CALLBACK -DHAVE_RL_CATCH_SIGNAL -DHAVE_RL_COMPLETION_APPEND_CHARACTER -DHAVE_RL_COMPLETION_DISPLAY_MATCHES_HOOK -DHAVE_RL_COMPLETION_MATCHES -DHAVE_RL_COMPLETION_SUPPRESS_APPEND -DHAVE_RL_PRE_INPUT_HOOK -I. -I/Users/michaelkarrer/.pyenv/versions/2.7.16/include/python2.7 -c Modules/2.x/readline.c -o build/temp.macosx-10.14-x86_64-2.7/Modules/2.x/readline.o -Wno-strict-prototypes
    creating build/lib.macosx-10.14-x86_64-2.7
    clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/michaelkarrer/.pyenv/versions/2.7.16/lib build/temp.macosx-10.14-x86_64-2.7/Modules/2.x/readline.o readline/libreadline.a readline/libhistory.a -lncurses -o build/lib.macosx-10.14-x86_64-2.7/readline.so
    clang: error: no such file or directory: 'readline/libreadline.a'
    clang: error: no such file or directory: 'readline/libhistory.a'
    error: command 'clang' failed with exit status 1

I did a full install of XCode but it did not change anything?!?

Since -L/usr/local/opt/readline/lib is in the compiler flags and the two files are under /usr/local/opt/readline/lib i have no clue why clang can not find those files?!?

Any help or clue would be really appreciated!

@anilsg

This comment has been minimized.

Copy link

commented Jul 16, 2019

@michaelkarrer81 , I found I could not pip install readline either. The compilation fails and therefore the library files are not built. That's why the linker can't find them. I used brew install readline instead and created links to the libraries from that (brew link) but I haven't tested it yet ...

@michaelkarrer81

This comment has been minimized.

Copy link

commented Jul 16, 2019

After a lot more hours of testing if found a way that at least worked for me:

  • Install complete Xcode (maybe not needed?)
  • Install Command Line Tools via App (With License Dialogue Pop Up) I did use the non beta with success but maybe just because is had beta 2 installed at some point? I could not build python with pyenv before "Command Line Tools Beta 2" but after removing it and replacing it with the non beta version i still can do pyenv install 2.7.16 without a problem
  • brew reinstall readline, xz, libxml2, libxmlsec1, python@2, pyenv, pyenv-virtualenv, ...
  • Add Package Path for all custom brew applications in .bash_profile
# PYTHON PYENV PREREQUISITES
# ==========================
# ATTENTION: You need to set the PKG_CONFIG_PATH to make "pip install ..." use the right locations! Conmpiler Flags are not necessary.

# Xcode SDK Root (only needed if messed up) 
# export SDKROOT="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14"

# openssl
# -------
export PATH="/usr/local/opt/openssl/bin:$PATH"
#export LDFLAGS="-L/usr/local/opt/openssl/lib $LDFLAGS"
#export CPPFLAGS="-I/usr/local/opt/openssl/include $CPPFLAGS"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig:$PKG_CONFIG_PATH"

# readline
# --------
#export LDFLAGS="-L/usr/local/opt/readline/lib $LDFLAGS"
#export CPPFLAGS="-I/usr/local/opt/readline/include $CPPFLAGS"
export PKG_CONFIG_PATH="/usr/local/opt/readline/lib/pkgconfig:$PKG_CONFIG_PATH"

# sqlite
# ------
export PATH="/usr/local/opt/sqlite/bin:$PATH"
#export LDFLAGS="-L/usr/local/opt/sqlite/lib $LDFLAGS"
#export CPPFLAGS="-I/usr/local/opt/sqlite/include $PKG_CONFIG_PATH"

# libxml2
# -------
export PATH="/usr/local/opt/libxml2/bin:$PATH"
#export LDFLAGS="-L/usr/local/opt/libxml2/lib"
#export CPPFLAGS="-I/usr/local/opt/libxml2/include"
export PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig:$PKG_CONFIG_PATH"

# libffi
# ------
#export LDFLAGS="-L/usr/local/opt/libffi/lib"
export PKG_CONFIG_PATH="/usr/local/opt/libffi/lib/pkgconfig:$PKG_CONFIG_PATH"

# pyenv
if command -v pyenv 1>/dev/null 2>&1; then
 eval "$(pyenv init -)"
fi

# pyenv pyenv-virtualenv
if which pyenv-virtualenv-init > /dev/null; then eval "$(pyenv virtualenv-init -)"; fi

After all of this i can successfully build python and install readline in the venv via pip pip install readline

Pain in the a...

@pablopunk

This comment has been minimized.

Copy link

commented Jul 16, 2019

FWIW the following let me install python 3.8-dev on macOS mojave:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.8-dev

I tried everything and this one was the only one that worked for me! 🎉

And I installed version 3.5.7 BTW

@anilsg

This comment has been minimized.

Copy link

commented Jul 17, 2019

@michaelkarrer81 , it seems more people are quoting success after installing Xcode itself. That might be the step that made the build possible for you.

@rgalanakis

This comment has been minimized.

Copy link

commented Jul 17, 2019

@pablopunk success with that (from a new shell window), thanks!!!

markosamuli added a commit to markosamuli/ansible-pyenv that referenced this issue Jul 19, 2019

fix: pyenv install on macOS Mojave
Find macOS headers in known Xcode application paths or Command Line Tools
directories.

Set MACOSX_DEPLOYMENT_TARGET and SDKROOT environment variables when
installing new Python versions as suggested in pyenv/pyenv#1066
discussion.

Use existing MACOSX_DEPLOYMENT_TARGET and SDKROOT environment variables
if available.

Use macOS release version from ansible_distribution_version as the value
for MACOSX_DEPLOYMENT_TARGET if not defined.

Fixes #16
@markosamuli

This comment has been minimized.

Copy link

commented Jul 19, 2019

Definining MACOSX_DEPLOYMENT_TARGET and SDKROOT environment variables seems to fix my previously failing pyenv install on Travis builds. I've tested it with Xcode 10.2.1 and Xcode 11.0 images.

With Xcode 11.0 I'm setting SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk but looking at the output, MacOSX10.15.sdk seems to be just a symbolic link to MacOSX.sdk in the same directory.

Xcode 10 release notes would suggest I could also try SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk if using CLT.

@Rotonen

This comment has been minimized.

Copy link

commented Jul 23, 2019

The combination of the latest macOS point release (10.14.6), the latest XCode release (10.3) and the latest Command Line Tools has remedied this for me.

@subhashb

This comment has been minimized.

Copy link

commented Jul 30, 2019

I had the same experience as @Rotonen ☝️
Just upgrading to macOS 10.14.6 fixed the problem.

@RonaldZhao

This comment has been minimized.

Copy link

commented Aug 7, 2019

I had the same problem with version 3.7.3.

The solution I found on the web is:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.7.3

thx. This is the only solution for me.

@NeilRoberts

This comment has been minimized.

Copy link

commented Aug 10, 2019

While setting up a spare MacBook from scratch (no data, so I can wipe and reinstall without much pain), I found that installing the macOS SDK headers results in the missing expat error when I try to install Python 3.x with pyenv (1.2.13, installed with homebrew). Starting over and skipping the header installation allows me to install a Python version.

My setup: fresh install of macOS 10.14.6, followed by the 10.14.6 supplemental update. Installed command line tools with xcode-select --install, install brew, pyenv, and pyenv-virtualenv. Neither SDKROOT nor CFLAGS are set. Xcode is not installed. /usr/include does not exist.

@elfotografo007

This comment has been minimized.

Copy link

commented Aug 13, 2019

The combination of the latest macOS point release (10.14.6), the latest XCode release (10.3) and the latest Command Line Tools has remedied this for me.

This was the clue to solve my problem. I was configuring a brand new Mac and also had the problem while installing 3.7.4. I had already installed the headers as other software required it. I had 10.14.6 and XCode 10.3. Downloading the latest Command Line Tools fixed the issue.

@tilgovi

This comment has been minimized.

Copy link

commented Aug 13, 2019

My previous comment aligns with what @NeilRoberts describes. Removing the headers package is not easy, but it does fix the problem. There is no longer any need to have the headers package installed for pyenv to build Python correctly.

I suspect / hope that perhaps the latest XCode uninstalls the headers, and that is why a number of folks here have reported that this fixes things. After all, Apple only published the headers package as a transitional package for software that depends on headers in the standard /usr/include location.

Can anyone who installed the SDK headers package and subsequently installed XCode 10.3 check whether or not they have a /usr/include directory?

@NeilRoberts

This comment has been minimized.

Copy link

commented Aug 14, 2019

I verified that I did not have a /usr/include directory, then installed the macOS SDK headers. After installation, /usr/include exists and is populated. pyenv install 3.6.9 fails with

ModuleNotFoundError: No module named 'pyexpat'

After installing Xcode 10.3 (from the App Store), launching it, and allowing it to install components, the /usr/include directory still exists and is populated and pyenv install 3.6.9 fails with the same error.

However, this worked after installing Xcode 10.3 (I did not try it after installing the headers, but before Xcode):

SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/ pyenv install 3.6.9

Misc other observations:

  • xcode-select -p returns /Applications/Xcode.app/Contents/Developer
  • After "reset to the default command line tools path", the previous command returns the same thing (I don't know what this was before installing Xcode).
@tilgovi

This comment has been minimized.

Copy link

commented Aug 14, 2019

Thanks, @NeilRoberts! Since some folks also seem to be reporting success with the XCode 11 Beta and you seem to have a spare machine to test these clean installs on, can you try installing XCode 11 Beta and see if that either fixes the problem, removes /usr/include, or both?

If we can conclude that things should Just Work going into the future, there may be nothing to do for this issue anymore.

@NeilRoberts

This comment has been minimized.

Copy link

commented Aug 14, 2019

I've started the download and will update this post tonight after I get the beta installed.

I've downloaded and installed Xcode 11.0 beta 5 (11M382q) and let it install its components. The /usr/include dir exists and is populated. pyenv install 3.5.4 again fails because it's unable to find pyexpat:

➜  ~ pyenv install 3.5.4
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.5.4.tar.xz...
-> https://www.python.org/ftp/python/3.5.4/Python-3.5.4.tar.xz
Installing Python-3.5.4...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 10.14.6 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/python-build.20190814204029.11806
Results logged to /var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/python-build.20190814204029.11806.log

Last 10 log lines:
    import pip
  File "/var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/tmp9l40c4tv/pip-9.0.1-py2.py3-none-any.whl/pip/__init__.py", line 26, in <module>
  File "/var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/tmp9l40c4tv/pip-9.0.1-py2.py3-none-any.whl/pip/utils/__init__.py", line 27, in <module>
  File "/var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/tmp9l40c4tv/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/pkg_resources/__init__.py", line 35, in <module>
  File "/private/var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/python-build.20190814204029.11806/Python-3.5.4/Lib/plistlib.py", line 65, in <module>
    from xml.parsers.expat import ParserCreate
  File "/private/var/folders/qn/lhhj_r5j2256_kynk30w3f1r0000gn/T/python-build.20190814204029.11806/Python-3.5.4/Lib/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ImportError: No module named 'pyexpat'
make: *** [install] Error 1

I've noticed a subtle change in the error, though I don't know if it's meaningful. The original error that I saw was:

ModuleNotFoundError: No module named 'pyexpat'

But the error that I've seen most recently is:

ImportError: No module named 'pyexpat'

Other things:

I tried changing the path for the active developer directory with sudo xcode-select -s /Applications/Xcode-beta.app/Contents/Developer, which yields the same error:

I thought it was odd that python-build is using lib from the Xcode SDK, so I checked my brew install and it has zlib, with this note:

For compilers to find zlib you may need to set:
  export LDFLAGS="-L/usr/local/opt/zlib/lib"
  export CPPFLAGS="-I/usr/local/opt/zlib/include"

I tried those exports and installing Python 3.5.4 (at this point I'm just picking different versions to avoid uninstalling a previous one), but I received the ImportError again. Does anyone know how to determine which version of lib is installed/active?

@tilgovi you are correct: I can burn this system to the ground to try anything new - just make a suggestion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.