cstddef not found when compiling C++ Extension - macOS

[Haydnspass]

🐛 Bug

I wanted to write a small C++ extension and compile it via setuptools beforehand as mentioned in the Tutorial. However there seems to be library issues on my mac. Can someone help?
The latest xcode command line tools are installed.

To Reproduce

Steps to reproduce the behavior:

1. write any C++ extension for pytorch
2. $ CC=clang CXX=clang++ NO_CUDA=1 python setup.py install
3. OR: $ NO_CUDA=1 python setup.py install

Error

Running:
$ CC=clang CXX=clang++ NO_CUDA=1 python setup.py install
Results in:
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' running install running bdist_egg running egg_info writing splitFrames.egg-info/PKG-INFO writing dependency_links to splitFrames.egg-info/dependency_links.txt writing top-level names to splitFrames.egg-info/top_level.txt reading manifest file 'splitFrames.egg-info/SOURCES.txt' writing manifest file 'splitFrames.egg-info/SOURCES.txt' installing library code to build/bdist.macosx-10.7-x86_64/egg running install_lib running build_ext building 'splitFrames' extension clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/lucasmueller/anaconda/envs/embl/include -arch x86_64 -I/Users/lucasmueller/anaconda/envs/embl/include -arch x86_64 -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/TH -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/THC -I/Users/lucasmueller/anaconda/envs/embl/include/python3.7m -c example-app.cpp -o build/temp.macosx-10.7-x86_64-3.7/example-app.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=splitFrames -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found] In file included from example-app.cpp:1: In file included from /Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include/torch/torch.h:3: In file included from /Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include/torch/all.h:3: /Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include/torch/cuda.h:5:10: fatal error: 'cstddef' file not found #include <cstddef> ^~~~~~~~~ 1 warning and 1 error generated. error: command 'clang' failed with exit status 1

However running:
$ NO_CUDA=1 python setup.py install
Also doesn't work, with the additional warning that the compiler is not compatible.

`No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running install
running bdist_egg
running egg_info
writing splitFrames.egg-info/PKG-INFO
writing dependency_links to splitFrames.egg-info/dependency_links.txt
writing top-level names to splitFrames.egg-info/top_level.txt
reading manifest file 'splitFrames.egg-info/SOURCES.txt'
writing manifest file 'splitFrames.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_ext
/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/utils/cpp_extension.py:166: UserWarning:

                           !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++) is not compatible with the compiler Pytorch was
built with for this platform, which is clang++ on darwin. Please
use clang++ to to compile your extension. Alternatively, you may
compile PyTorch from source using g++, and then you can also use
g++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                          !! WARNING !!

platform=sys.platform))
building 'splitFrames' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/lucasmueller/anaconda/envs/embl/include -arch x86_64 -I/Users/lucasmueller/anaconda/envs/embl/include -arch x86_64 -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/TH -I/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/THC -I/Users/lucasmueller/anaconda/envs/embl/include/python3.7m -c example-app.cpp -o build/temp.macosx-10.7-x86_64-3.7/example-app.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=splitFrames -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead
[-Wstdlibcxx-not-found]
In file included from example-app.cpp:1:
In file included from /Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include/torch/torch.h:3:
In file included from /Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include/torch/all.h:3:
/Users/lucasmueller/anaconda/envs/embl/lib/python3.7/site-packages/torch/lib/include/torch/csrc/api/include/torch/cuda.h:5:10: fatal error: 'cstddef' file
not found
#include
^~~~~~~~~
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1`

Environment

PyTorch version: 1.0.1
Is debug build: No
CUDA used to build PyTorch: None

OS: Mac OSX 10.14.2
GCC version: Could not collect
CMake version: version 3.13.0

Python version: 3.7
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip] Could not collect
[conda] blas 1.0 mkl
[conda] gpytorch 0.1.1 pypi_0 pypi
[conda] mkl 2018.0.3 1
[conda] mkl_fft 1.0.6 py37hb8a8100_0
[conda] mkl_random 1.0.1 py37h5d10147_1
[conda] pytorch 1.0.1 py3.7_0 soumith
[conda] torchvision 0.2.1 py_2 soumith

[ezyang]

We debugged a similar issue in Roger-luo/pytorch-complex#5

There appears to be some problem with extension building on macs on master. cc @goldsborough @yf225

[fab-jul]

Same for me, with PyTorch 0.4 and 1.0. Any updates?

From other threads (e.g. here), it seems this might be due to OSX 10.14. Same file compiles on Debian for me.

[Haydnspass]

@fab-jul the fix of @williamFalcon (Roger-luo/pytorch-complex#5) worked, however, I have a similar unsolved error after the Xcode / Mojave update last week.

@williamFalcon do you already have a solution for this?

clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
ld: library not found for -lstdc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'clang++' failed with exit status 1

[truh]

CFLAGS='-stdlib=libc++' python setup.py build

solved this problem for me.

[ezyang]

Thanks @truh, that's very useful information

[navneet-nmk]

For me, the solution provided by @truh did not work.
The following seems to work:

MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install

[Vincentqyw]

CFLAGS='-stdlib=libc++' python setup.py build

solved this problem for me.

thx!!! solved!

[admercs]

Also, be sure to use MacOS clang and not LLVM clang...

[zhangwenwen]

CFLAGS='-stdlib=libc++' python setup.py build

solved this problem for me.

It works for me, thx!