Skip to content

build failure with Xcode 10 - libstdc++ not supported anymore #23424

Description

@robbuckley

Code Sample, a copy-pastable example if possible

(pandas-dev) bilbo:pandas-rb robertbuckley$ python setup.py build_ext --inplace -j 4

running build_ext
building 'pandas._libs.window' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include -arch x86_64 -Ipandas/_libs -I./pandas/_libs -Ipandas/_libs/src/klib -Ipandas/_libs/src -I/Users/robertbuckley/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/numpy/core/include -I/Users/robertbuckley/anaconda3/envs/pandas-dev/include/python3.6m -c pandas/_libs/window.cpp -o build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o -Wno-unused-function
warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
pandas/_libs/window.cpp:655:10: fatal error: 'ios' file not found
#include "ios"
         ^~~~~
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1

Problem description

I setup a dev environment on Mac OS 10.13 according to https://pandas.pydata.org/pandas-docs/stable/contributing.html#creating-a-development-environment and all was well

Upgraded to 10.14 (mojave) and Xcode 10, and now i get the above failure.

Xcode 10 release notes includes the following deprecation notice

Building with libstdc++ was deprecated with Xcode 8 and is not supported in Xcode 10 when targeting iOS. C++ projects must now migrate to libc++ and are recommended to set a deployment target of macOS 10.9 or later, or iOS 7 or later. Besides changing the C++ Standard Library build setting, developers should audit hard-coded linker flags and target dependencies to remove references to libstdc++ (including -lstdc++, -lstdc++.6.0.9, libstdc++.6.0.9.tbd, and libstdc++.6.0.9.dylib). Project dependencies such as static archives that were built against libstdc++ will also need to be rebuilt against libc++. (40885260)

Some workaround suggestions are at https://stackoverflow.com/questions/52425766/stdlibc-headers-not-found-error-on-xcode-10 and the answer its linked to.

I dont have access to Xcode 9 anymore, so i can't try the workaround of copying the old libs from there

Activity

  1. TomAugspurger commented on Oct 30, 2018

    @TomAugspurger
    Contributor

    Another contributor fixed that with xcode select --install. Does that work for you?

  2. WillAyd commented on Oct 30, 2018

    @WillAyd
    Member

    Couldn't reproduce this on Mojave with Xcode 10.0 - let us know if the above suggestion doesn't fix it

  3. robbuckley commented on Oct 31, 2018

    @robbuckley
    ContributorAuthor

    @TomAugspurger i already did that, to fix another issue. Now when I enter xcode-select --install i get in response:
    xcode-select: error: command line tools are already installed, use "Software Update" to install updates

    I've updated to xcode 10.1, the issue is still there. Tomorrow I'll try to install Xcode 9 again.

    Might there be a case for moving to libc++ anyway ?

  4. robbuckley commented on Nov 1, 2018

    @robbuckley
    ContributorAuthor

    hi,

    i managed to get it building by deleting Xcode, and dowloading + installing Xcode 9.4.1 Command Line Tools from the Apple Developer website.

    There is still something funny going on, because I then installed Xcode 10 Command Line Tools (after first renaming the old ones at /Library/Developer/CommandLineTools). While I could use sudo xcode-select -s <path> to switch between the 2 versions, and clang --version confirmed it was switching the compiler version at least, I am suspicious that the libstdc++ libs and headers from 9.4.1 have been left somewhere thats not affected by the switch. So there may be some scenarios where this breaks (eg fresh install of Mojave).

    I'll continue to investigate, if thats OK

  5. JustinZhengBC commented on Nov 3, 2018

    @JustinZhengBC
    Contributor

    Not sure if it's the same issue or if this is a good fix, but I also couldn't build after updating to mojave (I got my error at a different point though). I fixed it by adding the argument extra_link_args=["-stdlib=libc++", "-mmacosx-version-min=10.9"] to the initialization of Extension object obj here.

    pandas/setup.py

    Lines 672 to 680 in d78bd7a

    obj = Extension('pandas.{name}'.format(name=name),
    sources=sources,
    depends=data.get('depends', []),
    include_dirs=include,
    language=data.get('language', 'c'),
    define_macros=data.get('macros', macros),
    extra_compile_args=extra_compile_args)
    extensions.append(obj)

    Credit for idea here

    Here's my error log before performing the above step:

    g++ -bundle -undefined dynamic_lookup -L/Users/admin/anaconda3/envs/pandas-dev/lib -arch x86_64 -L/Users/admin/anaconda3/envs/pandas-dev/lib -arch x86_64 -L/usr/local/opt/libarchive/lib -I/usr/local/opt/libarchive/include -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/pandas/_libs/window.o -o /Users/admin/Documents/Projects/pandas-jz/pandas/_libs/window.cpython-36m-darwin.so
    clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
    ld: library not found for -lstdc++
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command 'g++' failed with exit status 1
    
  6. TomAugspurger commented on Nov 5, 2018

    @TomAugspurger
    Contributor

    cython/cython#2694 also suggests that upgrading the command line tools was sufficient.

    @JustinZhengBC, did you get the same error (without the extra link args) in a clean environment? Or was this in the same environment (with previously compiled files) from before you upgraded your command line tools?

  7. JustinZhengBC commented on Nov 5, 2018

    @JustinZhengBC
    Contributor

    Yes, I get the error after running python setup.py clean as well as in a fresh download. I have reinstalled the dependencies from ci/requirements_dev.txt, and I am using the latest version of XCode and command line tools as suggested in that thread.

  8. TomAugspurger commented on Nov 5, 2018

    @TomAugspurger
    Contributor

    Thanks, that's strange.

    I'm just hesitant to add extra arguments to our setup script when we (I) don't really understand the root problem and why adding those arguments fixes it.

  9. WillAyd commented on Nov 5, 2018

    @WillAyd
    Member

    @JustinZhengBC any chance you installed some of the components via homebrew as well? Shot in the dark but I'm wondering if there could be some type of conflict there between that installation alongside the standard Xcode-select install

  10. JustinZhengBC commented on Nov 5, 2018

    @JustinZhengBC
    Contributor

    @WillAyd yes, I use homebrew and some c++ libraries do show up in brew list

  11. WillAyd commented on Nov 5, 2018

    @WillAyd
    Member

    Wonder if that could be it. There's also a page for homebrew dedicated to the topic:

    https://docs.brew.sh/C++-Standard-Libraries

    Do you have a way of either removing or upgrading those installed libraries to see if that resolves?

  12. JustinZhengBC commented on Nov 5, 2018

    @JustinZhengBC
    Contributor

    I tried renaming the Cellar folder and running build, but it still failed.

    However, I did manage to get it building without extra arguments after running /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg as described here. My build is working as intended now.

  13. WillAyd commented on Nov 5, 2018

    @WillAyd
    Member

    Hmm that's interesting. I wonder if that is part of the "Additional Components" that you get prompted to install when opening Xcode - any chance you recall having installed that or not?

  14. JustinZhengBC commented on Nov 5, 2018

    @JustinZhengBC
    Contributor

    I just ran through the installer again, no such prompt appeared. I've opened XCode to look at the settings but I haven't opened any projects in it since updating, so I might've missed the prompt there

  15. 6 remaining items

  16. robbuckley commented on Dec 11, 2018

    @robbuckley
    ContributorAuthor

    Hi, I got to the bottom of this issue, in the end, after a trawl through clang, distutils and python make/config files.

    The compiler standard library defaults to either libstdc++ or libc++, depending on the targetted macOS version - libstdc++ for 10.8 and below, and libc++ for 10.9 and above. This is determined by the environment variable MACOSX_DEPLOYMENT_TARGET or the compiler option-mmacosx-version-min, defaulting to the system version otherwise.

    When distuils builds extensions on macOS, it setsMACOSX_DEPLOYMENT_TARGET to the version that python was compiled with, even if the host system / Xcode is newer.

    Recent macOS versions of python have a 64-bit only variant built for 10.9 (python.org), and a universal 64/32-bit variant built for 10.6 (python.org) or 10.7 (conda). I am running the conda universal variant, so distutils targets macOS 10.7, despite my system being 10.14, with Xcode 10 which doesn't install libstdc++. Hence the failure I saw.

    It may work for some users, for 1 of 2 reasons:

    • libstdc++ is lying around somewhere from a previous Xcode 9 install
    • when running a recent 64-bit only variant of python, which targets macOS 10.9 and hence libc++

    I've pushed a revised change which sets the targeted macOS to 10.9, when running on a 10.9 or above system, with a python/distutils which targets pre-10.9. This should not break builds on pre-10.9 systems, unlike my previous approach. It fixes the issue on my system. CI is running now.

    Please let me know if you'd like me to submit it as a PR (assuming CI passes)

  17. JimJeon commented on Dec 16, 2018

    @JimJeon
    Contributor

    @robbuckley I experienced the same issue. Thanks for the PR!

  18. robbuckley commented on Dec 20, 2018

    @robbuckley
    ContributorAuthor

    @WillAyd could you please remove the 'cant repro tag'?

  19. added
    BuildLibrary building on various platforms
    and removed on Dec 20, 2018
  20. added this to the 0.24.0 milestone on Dec 28, 2018
  21. changed the title [-]build failure with Xcode 10 - stdlibc++ not supported anymore[/-] [+]build failure with Xcode 10 - libstdc++ not supported anymore[/+] on Jan 5, 2019
  22. OswaldoBornemann commented on Dec 12, 2019

    @OswaldoBornemann

    Hey, I got the same errors but could not find /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg on macOS Catalina Version 10.15.2 with Xcode 11.2.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BuildLibrary building on various platforms

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions