New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build failure with Xcode 10 - libstdc++ not supported anymore #23424

Closed
robbuckley opened this Issue Oct 30, 2018 · 22 comments

Comments

Projects
None yet
6 participants
@robbuckley
Copy link
Contributor

robbuckley commented Oct 30, 2018

Code Sample, a copy-pastable example if possible

(pandas-dev) bilbo:pandas-rb robertbuckley$ python setup.py build_ext --inplace -j 4

running build_ext
building 'pandas._libs.window' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -Ipandas/_libs -I./pandas/_libs -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include/python3.6m -c pandas/_libs/window.cpp -o build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o -Wno-unused-function
warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
pandas/_libs/window.cpp:655:10: fatal error: 'ios' file not found
#include "ios"
         ^~~~~
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1

Problem description

I setup a dev environment on Mac OS 10.13 according to https://pandas.pydata.org/pandas-docs/stable/contributing.html#creating-a-development-environment and all was well

Upgraded to 10.14 (mojave) and Xcode 10, and now i get the above failure.

Xcode 10 release notes includes the following deprecation notice

Building with libstdc++ was deprecated with Xcode 8 and is not supported in Xcode 10 when targeting iOS. C++ projects must now migrate to libc++ and are recommended to set a deployment target of macOS 10.9 or later, or iOS 7 or later. Besides changing the C++ Standard Library build setting, developers should audit hard-coded linker flags and target dependencies to remove references to libstdc++ (including -lstdc++, -lstdc++.6.0.9, libstdc++.6.0.9.tbd, and libstdc++.6.0.9.dylib). Project dependencies such as static archives that were built against libstdc++ will also need to be rebuilt against libc++. (40885260)

Some workaround suggestions are at https://stackoverflow.com/questions/52425766/stdlibc-headers-not-found-error-on-xcode-10 and the answer its linked to.

I dont have access to Xcode 9 anymore, so i can't try the workaround of copying the old libs from there

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Oct 30, 2018

Another contributor fixed that with xcode select --install. Does that work for you?

@WillAyd WillAyd added the Can't Repro label Oct 30, 2018

@WillAyd

This comment has been minimized.

Copy link
Member

WillAyd commented Oct 30, 2018

Couldn't reproduce this on Mojave with Xcode 10.0 - let us know if the above suggestion doesn't fix it

@robbuckley

This comment has been minimized.

Copy link
Contributor Author

robbuckley commented Oct 31, 2018

@TomAugspurger i already did that, to fix another issue. Now when I enter xcode-select --install i get in response:
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

I've updated to xcode 10.1, the issue is still there. Tomorrow I'll try to install Xcode 9 again.

Might there be a case for moving to libc++ anyway ?

@robbuckley

This comment has been minimized.

Copy link
Contributor Author

robbuckley commented Nov 1, 2018

hi,

i managed to get it building by deleting Xcode, and dowloading + installing Xcode 9.4.1 Command Line Tools from the Apple Developer website.

There is still something funny going on, because I then installed Xcode 10 Command Line Tools (after first renaming the old ones at /Library/Developer/CommandLineTools). While I could use sudo xcode-select -s <path> to switch between the 2 versions, and clang --version confirmed it was switching the compiler version at least, I am suspicious that the libstdc++ libs and headers from 9.4.1 have been left somewhere thats not affected by the switch. So there may be some scenarios where this breaks (eg fresh install of Mojave).

I'll continue to investigate, if thats OK

@JustinZhengBC

This comment has been minimized.

Copy link
Contributor

JustinZhengBC commented Nov 3, 2018

Not sure if it's the same issue or if this is a good fix, but I also couldn't build after updating to mojave (I got my error at a different point though). I fixed it by adding the argument extra_link_args=["-stdlib=libc++", "-mmacosx-version-min=10.9"] to the initialization of Extension object obj here.

pandas/setup.py

Lines 672 to 680 in d78bd7a

obj = Extension('pandas.{name}'.format(name=name),
sources=sources,
depends=data.get('depends', []),
include_dirs=include,
language=data.get('language', 'c'),
define_macros=data.get('macros', macros),
extra_compile_args=extra_compile_args)
extensions.append(obj)

Credit for idea here

Here's my error log before performing the above step:

g++ -bundle -undefined dynamic_lookup -L/Users/admin/anaconda3/envs/pandas-dev/lib -arch x86_64 -L/Users/admin/anaconda3/envs/pandas-dev/lib -arch x86_64 -L/usr/local/opt/libarchive/lib -I/usr/local/opt/libarchive/include -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o -o /Users/admin/Documents/Projects/pandas-jz/pandas/_libs/window.cpython-36m-darwin.so
clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
ld: library not found for -lstdc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'g++' failed with exit status 1
@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Nov 5, 2018

cython/cython#2694 also suggests that upgrading the command line tools was sufficient.

@JustinZhengBC, did you get the same error (without the extra link args) in a clean environment? Or was this in the same environment (with previously compiled files) from before you upgraded your command line tools?

@JustinZhengBC

This comment has been minimized.

Copy link
Contributor

JustinZhengBC commented Nov 5, 2018

Yes, I get the error after running python setup.py clean as well as in a fresh download. I have reinstalled the dependencies from ci/requirements_dev.txt, and I am using the latest version of XCode and command line tools as suggested in that thread.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Nov 5, 2018

Thanks, that's strange.

I'm just hesitant to add extra arguments to our setup script when we (I) don't really understand the root problem and why adding those arguments fixes it.

@WillAyd

This comment has been minimized.

Copy link
Member

WillAyd commented Nov 5, 2018

@JustinZhengBC any chance you installed some of the components via homebrew as well? Shot in the dark but I'm wondering if there could be some type of conflict there between that installation alongside the standard Xcode-select install

@JustinZhengBC

This comment has been minimized.

Copy link
Contributor

JustinZhengBC commented Nov 5, 2018

@WillAyd yes, I use homebrew and some c++ libraries do show up in brew list

@WillAyd

This comment has been minimized.

Copy link
Member

WillAyd commented Nov 5, 2018

Wonder if that could be it. There's also a page for homebrew dedicated to the topic:

https://docs.brew.sh/C++-Standard-Libraries

Do you have a way of either removing or upgrading those installed libraries to see if that resolves?

@JustinZhengBC

This comment has been minimized.

Copy link
Contributor

JustinZhengBC commented Nov 5, 2018

I tried renaming the Cellar folder and running build, but it still failed.

However, I did manage to get it building without extra arguments after running /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg as described here. My build is working as intended now.

@WillAyd

This comment has been minimized.

Copy link
Member

WillAyd commented Nov 5, 2018

Hmm that's interesting. I wonder if that is part of the "Additional Components" that you get prompted to install when opening Xcode - any chance you recall having installed that or not?

@JustinZhengBC

This comment has been minimized.

Copy link
Contributor

JustinZhengBC commented Nov 5, 2018

I just ran through the installer again, no such prompt appeared. I've opened XCode to look at the settings but I haven't opened any projects in it since updating, so I might've missed the prompt there

@WillAyd

This comment has been minimized.

Copy link
Member

WillAyd commented Nov 5, 2018

I believe if you uninstall and reinstall Xcode it should prompt you for additional components the first time you open it (no project required).

Something does seem off here; if you could help pinpoint would be great. May need some kind of doc updates

@JustinZhengBC

This comment has been minimized.

Copy link
Contributor

JustinZhengBC commented Nov 6, 2018

A fresh install of the latest Xcode and command line tools can build with no errors and no prompts. However, while I can get old versions of Xcode from the Apple developer's website, those seem to be standalones so I can't replicate the update process.

@robbuckley

This comment has been minimized.

Copy link
Contributor Author

robbuckley commented Nov 7, 2018

i managed to repro the issue on my system with the following sequence of steps:

  1. uninstalling Xcode.app and command line tools (rm -rf /Library/Developer)
  2. removing /usr/include. This needs ’System Integrity Protection’ to be disabled, see (https://superuser.com/questions/995360/missing-usr-include-in-os-x-el-capitan#1009364)
  3. install ‘Command Line Tools (mac OS 10.14) for Xcode 10.1 from https://developer.apple.com/download/more/ (needs a [free] apple developer login)

After this I could reproduce the original failure mode i reported, and could get the extensions compiling OK using a variant of the fix proposed by @JustinZhengBC:

obj = Extension('pandas.{name}'.format(name=name),
                sources=sources,
                depends=data.get('depends', []),
                include_dirs=include,
                language=data.get('language', 'c'),
                define_macros=data.get('macros', macros),
                extra_compile_args=extra_compile_args + ['-stdlib=libc++'],
                extra_link_args=['-stdlib=libc++']

Probably these changes should be wrapped with a check for Mac OS >= 10.9 to avoid breaking other platforms.

If i added only the part in extra_link_args, I see the compiler error i originally reported:

building 'pandas._libs.window' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -Ipandas/_libs -I./pandas/_libs -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include/python3.6m -c pandas/_libs/window.cpp -o build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o
warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead
      [-Wstdlibcxx-not-found]
pandas/_libs/window.cpp:655:10: fatal error: 'ios' file not found
#include "ios"
         ^~~~~
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1

If i added only the extra_compile_args part, the compile succeeds but the linker would throw an error:

building 'pandas._libs.window' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -Ipandas/_libs -I./pandas/_libs -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include/python3.6m -c pandas/_libs/window.cpp -o build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o -Wno-unused-function -stdlib=libc++
In file included from pandas/_libs/window.cpp:663:
In file included from /Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18:
In file included from /Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1823:
/Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: 
      "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it by " \
 ^
pandas/_libs/window.cpp:45351:27: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'Py_ssize_t' (aka 'long')
      [-Wsign-compare]
  __pyx_t_2 = ((__pyx_t_1 != __pyx_v_N) != 0);
                ~~~~~~~~~ ^  ~~~~~~~~~
2 warnings generated.
g++ -bundle -undefined dynamic_lookup -L/Users/robertbuckley/anaconda3/envs/pandas-dev/lib -arch x86_64 -L/Users/robertbuckley/anaconda3/envs/pandas-dev/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o -o /Users/robertbuckley/python-scripts/pandas-rb/pandas/_libs/window.cpython-36m-darwin.so
clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
ld: library not found for -lstdc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'g++' failed with exit status 1

My working theory is that the older version of Xcode command line tools adds the headers into /usr/include and the libstdc++ binaries into somewhere like /usr/lib, and that subsequent uninstalls of Xcode / Command Line Tools don’t remove them.

possibly useful liinks

libc++ https://libcxx.llvm.org/
LLVM (clang) changes where the warning stdlibcxx-not-found was added https://reviews.llvm.org/D48297
extensive discussion on conda-forge conda-forge/conda-forge.github.io#183

@robbuckley

This comment has been minimized.

Copy link
Contributor Author

robbuckley commented Nov 28, 2018

hi @WillAyd the behaviour seems to depend if clang is able to find stdlibc++ headers/libs. This depends on the xcode install history, hence the difficulty in reproducing this.

i suppose there are 2 options: (1) require the user to workaround this by installing stdlibc++ or (2) change mac os targets to build + link libc++.

So far i am unable to get (1) working again on my system. I've tried:

  • xcode-select--install, this responds with xcode-select: error: command line tools are already installed, use "Software Update" to install updates.
  • installing /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg as per @JustinZhengBC

But after each step I get the original compile error I reported.

for option 2 i've created a PR. I didn't meant to preempt any decision on taking this - its my first PR and i didnt realise that pushing a branch to my fork would update the GH issue. CI is running now with travis, azure (passed), circle ci, appveyor (passed)

please let me know how you'd like to proceed

@WillAyd

This comment has been minimized.

Copy link
Member

WillAyd commented Nov 28, 2018

Hmm is our only requirement for the C++ headers/libs stemming from this?

#19549 (comment)

Not saying it's worth changing here but curious if we previously looked at the performance difference in linking against the c++ implementation vs using collections.deque in Cython and if the latter wouldn't simplify linkage

@chris-b1 @jreback

@robbuckley

This comment has been minimized.

Copy link
Contributor Author

robbuckley commented Dec 11, 2018

Hi, I got to the bottom of this issue, in the end, after a trawl through clang, distutils and python make/config files.

The compiler standard library defaults to either libstdc++ or libc++, depending on the targetted macOS version - libstdc++ for 10.8 and below, and libc++ for 10.9 and above. This is determined by the environment variable MACOSX_DEPLOYMENT_TARGET or the compiler option-mmacosx-version-min, defaulting to the system version otherwise.

When distuils builds extensions on macOS, it setsMACOSX_DEPLOYMENT_TARGET to the version that python was compiled with, even if the host system / Xcode is newer.

Recent macOS versions of python have a 64-bit only variant built for 10.9 (python.org), and a universal 64/32-bit variant built for 10.6 (python.org) or 10.7 (conda). I am running the conda universal variant, so distutils targets macOS 10.7, despite my system being 10.14, with Xcode 10 which doesn't install libstdc++. Hence the failure I saw.

It may work for some users, for 1 of 2 reasons:

  • libstdc++ is lying around somewhere from a previous Xcode 9 install
  • when running a recent 64-bit only variant of python, which targets macOS 10.9 and hence libc++

I've pushed a revised change which sets the targeted macOS to 10.9, when running on a 10.9 or above system, with a python/distutils which targets pre-10.9. This should not break builds on pre-10.9 systems, unlike my previous approach. It fixes the issue on my system. CI is running now.

Please let me know if you'd like me to submit it as a PR (assuming CI passes)

@JimJeon

This comment has been minimized.

Copy link
Contributor

JimJeon commented Dec 16, 2018

@robbuckley I experienced the same issue. Thanks for the PR!

@robbuckley

This comment has been minimized.

Copy link
Contributor Author

robbuckley commented Dec 20, 2018

@WillAyd could you please remove the 'cant repro tag'?

@WillAyd WillAyd added Build OSX and removed Can't Repro labels Dec 20, 2018

@jreback jreback added this to the 0.24.0 milestone Dec 28, 2018

@einarf einarf referenced this issue Dec 30, 2018

Closed

Reminder: make a release #246

0 of 2 tasks complete

@robbuckley robbuckley changed the title build failure with Xcode 10 - stdlibc++ not supported anymore build failure with Xcode 10 - libstdc++ not supported anymore Jan 5, 2019

@ibell ibell referenced this issue Jan 10, 2019

Closed

OSX 10.14 builds #1782

emesterhazy added a commit to emesterhazy/quickfix-package that referenced this issue Jan 22, 2019

Mac builds target 10.9 on newer systems
Sets the deployment target to 10.9 for newer systems so that
`distutils` uses `libc++` instead of `libstdc++`, which is not
available on Macs running 10.9 and above.

See:
pandas-dev/pandas#23424
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment