Following instructions to install gives compiler errors when building python on Ubuntu

[HaraldKorneliussen]

Hello,

I tried to follow the instructions in the readme to install. First, I used my existing anaconda environment. However, it failed during the build of pytorch, the linker complaining about missing -pthreads among other things. I did some searching and found that maybe the anaconda compilers weren't installed, so I tried to install them with

conda install gcc_linux-64 gxx_linux-64 gfortran_linux-64

However, this was apparently the wrong compilers, since the build immediately failed with two unrecognized command line options: -fstack-protector-strong and -fno-plt

After that I thought I'd try using a fresh miniconda environment rather than my existing anaconda environment, so I did it over with following the miniconda installation steps in the README too. This time, it fails with linker being unable to find -lgcc_s, which again sounds a lot like a missing compiler install. Trying to install them with the same command line as above gives the same error.

Installing FAIR's tools - which are fantastic once I get them to work, thank you! - appear to be extremely sensitive to C++ compiler versions. The same with nvidia's tools... I stranded on this problem last time I tried to install pytorch from source, however then the new nightly build distributions of pytorch saved me then.

If you could provide similar bleeding edge binaries for ONNX and Caffe2, that'd be terrific! If not, if the build files could make sure the right compiler was used (and give sensible error messages if not) that would be a big improvement too.

[theweiho]

We're exploring the use of a new combined PyTorch-Caffe2 package that should hopefully alleviate some installation/compilation difficulties (see #43 (comment)). We'll also try to look into what PyTorch/Caffe2/ONNX/fairseq are doing with their make files to see if we can pull some best practices/version checking from them - thanks for the suggestion!

[jmp84]

@HaraldKorneliussen , would you be able to retry the instructions now? They have been updated to use the PyTorch-Caffe2 package @theweiho mentions.

[jmp84]

@HaraldKorneliussen, feel free to reopen if you still have issues with install.

[HaraldKorneliussen]

Hello, sorry I didn't see this before. I followed the new instructions, it fails at the cmake step with the following error message:

CMake Error at /home/harald/anaconda3/share/cmake/Caffe2/public/cuda.cmake:376 (message):
  CUDA 9.1 is not compatible with GCC version >= 7.  Use the following option
  to use another version (for example):

    -DCUDA_HOST_COMPILER=/usr/bin/gcc-6

by the way, yes, it's cuda 9.1 installed here, I hope the minor version number doesn't change things. If I add the line they suggest, building works until the final linking step, when it fails with

[100%] Linking CXX executable translation_decoder
/home/harald/anaconda3/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.2.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: warning: libgomp.so.1, needed by /home/harald/anaconda3/lib/libcaffe2.so, not found (try using -rpath or -rpath-link)

and then a long list of missing symbols.

Again I try the suggestion of passing -rpath or -rpath-link (via make LDFLAGS="-rpath-links ..."), but this doesn't help. This file is available many places in my system

/home/harald/anaconda3/pkgs/gcc-4.8.5-7/lib/libgomp.so.1
/home/harald/anaconda3/pkgs/libgcc-4.8.5-2/lib/libgomp.so.1
/home/harald/anaconda3/pkgs/libgcc-ng-7.2.0-hdf63c60_3/lib/libgomp.so.1
/home/harald/anaconda3/pkgs/libgcc-ng-7.2.0-hdf63c60_3/x86_64-conda_cos6-linux-gnu/sysroot/lib/libgomp.so.1
/home/harald/anaconda3/x86_64-conda_cos6-linux-gnu/sysroot/lib/libgomp.so.1
/usr/lib/x86_64-linux-gnu/libgomp.so.1

but it seems the linker can't find any of these (maybe the compiler version matters?)

Update: I also tried specifying gcc-4.8 as the cuda host compiler to cmake, but this gives the same error

[theweiho]

Does pre-pending the correct lib dir to the LD_LIBRARY_PATH shell variable help? Like

export LD_LIBRARY_PATH="/some/dir/lib:${LD_LIBRARY_PATH}"

Otherwise, consider searching online for other linker tips/tricks.

Also, given that your default GCC version is >= 7, what's the fallback version you're using per the error message? GCC 4.8.5 would be too old to work for us - I think we'd need at least 4.9.2.

[HaraldKorneliussen]

I finally got this to work with gcc-6. Specifying -DCUDA_HOST_COMPILER=/usr/bin/gcc-6 was not enough, I needed to set gcc 6 as the default compiler too with run update-alternatives --config gcc.