compiling error on Mac OS

[waynezw0618]

Hi
I followed the instruction of the installation by install pyblinder11 first.
but during compiling, still get error message like :
-fPIC -c ../source/smarties_pybind11.cpp -o smarties_pybind11.o
../source/smarties_pybind11.cpp:10:10: fatal error: 'pybind11/pybind11.h' file not found
#include <pybind11/pybind11.h>

then I use python -m pip install pybind11 to install pybind11 as suggested here . I got the following error message during link
mkdir -p ../lib
mkdir -p ../lib
mpic++ -shared -fPIC -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -o ../lib/libsmarties.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o -L/usr/local/opt/openblas/lib/ -lopenblas
mpic++ -shared -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -Wl,-undefined,dynamic_lookup -o ../lib/smarties.cpython-37m-darwin.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o smarties_pybind11.o -L/usr/local/opt/openblas/lib/ -lopenblas
ld: unknown option: -platform_version
ld: unknown option: -platform_version
clang-10clang-10: : error: error: linker command failed with exit code 1 (use -v to see invocation)
linker command failed with exit code 1 (use -v to see invocation)
make: *** [py11] Error 1
make: *** Waiting for unfinished jobs....
make: *** [lib] Error 1

Thanks !

[novatig]

Hi,

What happens if you type "python3 -m pybind11 --includes" ?

[waynezw0618]

Hi novatig:
Thanks for replying.

(python3) my-mbp:makefiles my$ python3 -m pybind11 --include
-I/anaconda2/envs/python3/include/python3.7m -I/Users/my/.local/lib/python3.7/site-packages/pybind11/include

and it is now can find the pybind11.h as I use "python -m pip install pybind11"
but still some errors for linking.

2 warnings generated.
mkdir -p ../lib
mkdir -p ../lib
mpic++ -shared -fPIC -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -o ../lib/libsmarties.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o -L/usr/local/opt/openblas/lib/ -lopenblas
mpic++ -shared -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -Wl,-undefined,dynamic_lookup -o ../lib/smarties.cpython-37m-darwin.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o smarties_pybind11.o -L/usr/local/opt/openblas/lib/ -lopenblas
ld: unknown option: -platform_version
ld: unknown option: -platform_version
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [py11] Error 1
make: *** Waiting for unfinished jobs....
make: *** [lib] Error 1

[novatig]

I see.
Try to compile with cmake (same commands as for linux).
Do you get the same error?

[waynezw0618]

Hi, Novatig
thanks very much for replying
I follow the that part and use cmake, still does not work me . here is there error message:

(python3) weis-mbp:build my$ /Applications/CMake.app/Contents/bin/cmake ../
-- The C compiler identification is AppleClang 10.0.1.10010046
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /usr/local/Cellar/open-mpi/4.0.3/lib/libmpi.dylib (found version "3.1")
-- Found MPI_CXX: /usr/local/Cellar/open-mpi/4.0.3/lib/libmpi.dylib (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found OpenMP_C: -Xclang -fopenmp (found version "3.1")
-- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
-- Found OpenMP: TRUE (found version "3.1")
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Looking for dgemm_
-- Looking for dgemm_ - found
-- Found BLAS: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Accelerate.framework
CMake Error at CMakeLists.txt:138 (add_subdirectory):
The source directory

/Users/my/software/ML/smarties/source/extern/pybind11

does not contain a CMakeLists.txt file.

[novatig]

It should: did you git clone --recursive https://github.com/cselab/smarties.git? Or just git clone?

[waynezw0618]

I used latter. you are right, I can now compile with former.
but still error messages from compiling the application
(python3) weis-mbp:cart_pole_cpp weizhang$ make
mpic++ -std=c++14 -Wall -O2 -I/Users/my/software/ML/smarties//include -Wextra -Wfloat-equal -Wundef -Wcast-align -Wno-unused-parameter -Wwrite-strings -Wlogical-op -Wmissing-declarations -Wredundant-decls -Wshadow -Woverloaded-virtual -Wuninitialized -Wno-div-by-zero -c cart-pole.cpp -o cart-pole.o
warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
In file included from cart-pole.cpp:9:
In file included from /Users/my/software/ML/smarties//include/smarties.h:12:
In file included from /Users/my/software/ML/smarties//include/../source/Communicator.h:12:
In file included from /Users/my/software/ML/smarties//include/../source/Core/Environment.h:12:
/Users/my/software/ML/smarties//include/../source/Core/Agent.h:228:14: warning: cast from 'char *' to 'smarties::episodeStatus *' increases required alignment from 1 to 4 [-Wcast-align]
return * (episodeStatus *) buffer;
^~~~~~~~~~~~~~~~~~~~~~~~
/Users/my/software/ML/smarties//include/../source/Core/Agent.h:233:14: warning: cast from 'char *' to 'smarties::learnerStatus *' increases required alignment from 1 to 4 [-Wcast-align]
return * (learnerStatus *) buffer;
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from cart-pole.cpp:9:
In file included from /Users/my/software/ML/smarties//include/smarties.h:12:
In file included from /Users/my/software/ML/smarties//include/../source/Communicator.h:13:
In file included from /Users/my/software/ML/smarties//include/../source/Utils/MPIUtilities.h:12:
/usr/local/Cellar/open-mpi/4.0.3/include/mpi.h:316:16: warning: 'STDC_VERSION' is not defined, evaluates to 0 [-Wundef]

elif (STDC_VERSION >= 201112L)

           ^

4 warnings generated.

mpic++ -o cart-pole cart-pole.o -L/Users/my/software/ML/smarties//lib -lsmarties
ld: unknown option: -platform_version
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [cart-pole] Error 1

[novatig]

Are you on OS X Mojave (Homebrew/homebrew-core#52461)?
Anyway, it seems your cmake works fine.
I changed from make to cmake, on the cart_pole_cpp app only (for now).
Pull again and see if it works.

[waynezw0618]

@novtig
perfect！it works now. can you please update other apps? I am interested in boatNav or glider

thanks

[novatig]

Hi, I'm very busy at the time and I won't stop being busy till the fall (I'm defending my PhD thesis in August).
If you urgently want to try those apps, have a read at the CMakeLists.txt and setup.sh files in apps/cart_pole_cpp/
They are really self-explanatory

[waynezw0618]

Sorry for troubling. I am trying to shift to ubuntu. still error arises like :

• The C compiler identification is GNU 6.3.0
  -- The CXX compiler identification is GNU 6.3.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  CMake Error at CMakeLists.txt:70 (target_compile_features):
  target_compile_features specified unknown feature "cxx_std_14" for target
  "libsmarties".

-- Found MPI_C: /usr/lib/x86_64-linux-gnu/libmpich.so
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/libmpichcxx.so;/usr/lib/x86_64-linux-gnu/libmpich.so
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- A library with BLAS API found.
-- Found PythonInterp: /usr/local/bin/python3.7 (found version "3.7.3")
-- Found PythonLibs: /usr/local/lib/libpython3.7m.so
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.4.dev4
-- Configuring incomplete, errors occurred!
See also "/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeOutput.log".
See also "/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeError.log".
here is information from CMAKEERROR
Determining if the pthread_create exist failed with the following output:
Change Dir: /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_0fad3/fast"
/usr/bin/make -f CMakeFiles/cmTC_0fad3.dir/build.make CMakeFiles/cmTC_0fad3.dir/build
make[1]: Entering directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o
/usr/bin/cc -o CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o -c /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
Linking C executable cmTC_0fad3
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_0fad3.dir/link.txt --verbose=1
/usr/bin/cc CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o -o cmTC_0fad3 -rdynamic
CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o: In function main': CheckSymbolExists.c:(.text+0x1b): undefined reference to pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_0fad3.dir/build.make:97: recipe for target 'cmTC_0fad3' failed
make[1]: *** [cmTC_0fad3] Error 1
make[1]: Leaving directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_0fad3/fast' failed
make: *** [cmTC_0fad3/fast] Error 2

File /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
(void)argv;
#ifndef pthread_create
return ((int*)(&pthread_create))[argc];
#else
(void)argc;
return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_8cb33/fast"
/usr/bin/make -f CMakeFiles/cmTC_8cb33.dir/build.make CMakeFiles/cmTC_8cb33.dir/build
make[1]: Entering directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_8cb33.dir/CheckFunctionExists.c.o
/usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_8cb33.dir/CheckFunctionExists.c.o -c /usr/share/cmake-3.7/Modules/CheckFunctionExists.c
Linking C executable cmTC_8cb33
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8cb33.dir/link.txt --verbose=1
/usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_8cb33.dir/CheckFunctionExists.c.o -o cmTC_8cb33 -rdynamic -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_8cb33.dir/build.make:97: recipe for target 'cmTC_8cb33' failed
make[1]: *** [cmTC_8cb33] Error 1
make[1]: Leaving directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_8cb33/fast' failed
make: *** [cmTC_8cb33/fast] Error 2

would you please help when you have time

[novatig]

This is not an issue with smarties. As always, just look at the first error and ask your best friend:
https://www.google.com/search?q=CMAKEERROR+Determining+if+the+pthread_create+exist+failed+with+the+following+output%3A

[waynezw0618]

Hi Novatig:
I understand that you are very busy at the moment. I am currently using your code. I am playing glider and boatNav. I am wondering, how to setup parameters in the json file, for instance, nnLayerSizes, epsAnneal and so on.
thanks for taking your time to answer the question.
Yours Sincerely

Wei

[novatig]

Hi, sorry for the wait.
Thank you for your message, it reminded me one way the README was (is?) lacking.
Try reading https://github.com/cselab/smarties/blob/master/README.rst#learner-settings-json-files and let me know any doubt you still have.

However, please open a new ticket and close this one if your doubts are unrelated to compilation.

[waynezw0618]

@novatig thanks! I have opened a new one (#4)