'omp.h' file not found when running setup.py in raisimGymTorch on macOS.

[sgillen]

Hello,

I've been experimenting with RaiSim over the last week, I appreciate the work you've put in and the permissive academic license. On linux I've gotten everything to work fine, however I have run into an issue when using raisimGymTorch on macOS. raisimLib builds fine and I can run examples, the issue is when I run "python setup.py develop" in raisimGymTorch.

The relevant part of the error message:

[ 33%] Building CXX object CMakeFiles/rsg_anymal.dir/raisimGymTorch/env/raisim_gym.cpp.o
In file included from /Users/sgillen/work/external/raisim/workspace/raisimLib/raisimGymTorch/raisimGymTorch/env/raisim_gym.cpp:8:
In file included from /Users/sgillen/work/external/raisim/workspace/raisimLib/thirdParty/pybind11/include/pybind11/eigen.h:36:
/usr/local/include/eigen3/Eigen/Core:266:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/rsg_anymal.dir/raisimGymTorch/env/raisim_gym.cpp.o] Error 1
make[1]: *** [CMakeFiles/rsg_anymal.dir/all] Error 2
make: *** [all] Error 2

The compiler being used here is the default apple clang for macOS 11.0.1 (this is confirmed by looking at CmakeCache, clang++ --version returns:

Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: x86_64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I don't think python is the problem, but this is with Python 3.6.9 installed via anaconda.

I've been seeing mixed messages on the state of openMP support with the built in MacOS utilities. Do you have any insight into this? Is there a dependency I am missing? Thanks very much.

[jhwangbo]

I pushed a fix. I didn't expect anyone will use raisimGym in Mac because there is no gpu support.

[sgillen]

Thanks, the fix seems to work for me. For some reason I needed to manually pass CMAKE_PREFIX_PATH for raisim when calling setup.py:

python setup.py --CMAKE_PREFIX_PATH {$LOCAL_INSTALL}/lib/cmake develop                     

And it gave me a warning about having two different versions of openMP installed, but I suspect that's a problem on my end.

Indeed I'm mostly interested in simply rendering rollouts from policies trained on a more powerful remote machine, although it seems I can do that with just raisimUnity? I haven't got that to work yet but I think I should be able to.

[jhwangbo]

Passing a CMAKE_PREIX_PATH is a way to do it if you copy the raisimGymTorch folder to other location. In that case, python doesnt know where you installed raisim (unless you installed it globally). Is that the case for you?

Duplicate omp issue seems ver common in Mac. I also had an error in running runner.py becasue of that. I added a line in VectorizedEnvironmnent.py to ignore it and it seems to work.

You can do it on pure cpp and raisimUnity but It will be more convenient to visualize it in raisimGym. So what you are doing makes sense

[sgillen]

No that's what is strange, I did not move raisimGymTorch, everything should be "standard" I used the same process that on my linux box resulted in cmake knowing where to find raisim automatically. Not a big deal for me though, just something I thought I'd mention, in case other people start seeing that too.

For duplicate omp I set KMP_DUPLICATE_LIB_OK=TRUE which cleared up that issue, hopefully that doesn't come back to bite me, as I said I don't plan to do any real training on the mac. (edit: this seems to be what you put into the VectorizedEnvironmnent.py too).

[sheim]

I'm having the same error when trying to run on an m1 mac. I've tried switching to brew-installed clang (since I installed eigen via brew, and based on this comment), but no luck. Any ideas?

[jhwangbo]

@sheim I have been digging into this problem for a while. For me, it fails to find OpenMP. Have you tried it with GCC? you can do that by defining CMAKE_CXX_COMPILER and CMAKE_C_COMPILER environment variables before you call python setup.py develop

[sheim]

Just tried, no dice.
But I can confirm the problem is finding OpenMP. I tried modifying VectorizedEnvironment.hpp and ../thirdParty/Eigen3/include/eigen3/Eigen/Core to look directly for the correct the file at <#include </usr/local/opt/libomp/include/omp.h> (super hacky, I know), which now solves finding the header but results in the following linking error:

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [../../raisimGymTorch/env/bin/rsg_anymal_debug_app] Error 1
make[1]: *** [CMakeFiles/rsg_anymal_debug_app.dir/all] Error 2

which seems to be trying to link through rosetta (x86_64)... but I haven't figured out why/where. That header file is in the arm64 version (brew arm64 goes to local/opt, brew x86 to /usr/local).
Tried this also with the brew-installed clang/gcc.

[jhwangbo]

This is so far what I have figured out

1. I was wrong. You have to CC and CXX variables to change the compiler
2. I had to change the cmake file of the raisimGymTorch to compile the python binary for arm64. You have to call target_compile_definition(${subdir} PRIVATE -mcpu=lightning). I could do that only thru the raw cmake command. When I go through the python setup.py, it still compiles the binary for x86
3. It turns out that my conda is compiled for x86. So python seems to overwrite the architecture to x86.
4. So I have to change the conda and try again.

I have a few impending deadlines. I'll look at it again once I have free time.

[sheim]

Thanks, and no rush, I'm also looking at this only on the weekend, and working on ubuntu otherwise.