Building version 1.3.2 fails with undefined reference errors

[samfux84]

Hi,

I am trying to build the connectome workbench 1.3.2 using GCC 4.8.5, Qt 5.9.1, Mesa 17.2.3 and CMake 3.11.1. The build almost finishes and then fails at 93% with the error message:

[ 93%] Building CXX object GuiQt/CMakeFiles/GuiQt.dir/KeyEvent.cxx.o
cd /scratch/93741177.tmpdir/workbench-1.3.2/src/build/GuiQt && /usr/bin/g++  -DCARET_DOTFCN -DCARET_OS_LINUX -DHAVE_FREETYPE -DQT_CORE_LIB -DQT_NO_DEBUG -I/scratch/93741177.tmpdir/workbench-1.3.2/src/kloewe/dot/src -I/cluster/apps/gcc-4.8.5/mesa-17.2.3-scgjhvf43ivc54ykewsggroiyydhdtma/include -I/cluster/apps/gcc-4.8.5/zlib-1.2.11-tu4n3a4rud6dtfqmgmgfrb32liysc5lu/include -isystem /cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/include -isystem /cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/include/QtCore -isystem /cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/./mkspecs/linux-g++ -I/cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/include/QtOpenGL -I/cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/include/QtWidgets -I/cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/include/QtGui -I/cluster/apps/gcc-4.8.5/qt-5.9.1-4xilvu6otyfkgeasjlf27tyyvplvz33d/include/QtNetwork -I/scratch/93741177.tmpdir/workbench-1.3.2/src/build/GuiQt -I/scratch/93741177.tmpdir/workbench-1.3.2/src/GuiQt -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Qwt -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Algorithms -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Annotations -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Brain -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Commands -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Charting -I/scratch/93741177.tmpdir/workbench-1.3.2/src/FilesBase -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Files -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Cifti -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Gifti -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Graphics -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Nifti -I/scratch/93741177.tmpdir/workbench-1.3.2/src/OSMesaDummy -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Operations -I/scratch/93741177.tmpdir/workbench-1.3.2/src/OperationsBase -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Palette -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Scenes -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Xml -I/scratch/93741177.tmpdir/workbench-1.3.2/src/Common  -fopenmp  -O3 -DNDEBUG -ftree-vectorize -march=core-avx2 -mavx2 -fPIC   -W -Wall -Werror=return-type -Werror=switch -Wunused-parameter -fPIC -std=gnu++11 -o CMakeFiles/GuiQt.dir/KeyEvent.cxx.o -c /scratch/93741177.tmpdir/workbench-1.3.2/src/GuiQt/KeyEvent.cxx
../kloewe/dot/libdot.a(dot.c.o): In function `dot_set_impl':
dot.c:(.text+0xf5): undefined reference to `sdot_avx'
dot.c:(.text+0x106): undefined reference to `ddot_avx'
dot.c:(.text+0x11e): undefined reference to `dsdot_avx'
collect2: error: ld returned 1 exit status
make[2]: *** [CommandLine/CMakeFiles/wb_command.dir/build.make:167: CommandLine/wb_command] Error 1
make[2]: Leaving directory '/scratch/93741177.tmpdir/workbench-1.3.2/src/build'
make[1]: *** [CMakeFiles/Makefile2:1467: CommandLine/CMakeFiles/wb_command.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Any help is appreciated. If there is any useful information that I can provide, then please let me know.

[sfux@eu-c7-041-01 build]$ cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
[sfux@eu-c7-041-01 build]$ uname -a
Linux eu-c7-041-01 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 GNU/Linux
[sfux@eu-c7-041-01 build]$

[johnvelab]

The "dot" code can be disabled:

• In your build directory, edit the CMakeCache.txt file,
• Search for WORKBENCH_USE_SIMD and set its value to FALSE
• save the file
• Rerun cmake in the build directory to use the updated CMakeCache.txt with the command "cmake ."
• Then run "make"

[coalsont]

I think the ifdef testing in the "dot" library is probably confused somehow, but it is a bit tricky for me to figure out what might be the cause. @kloewe ?

[coalsont]

FYI, KeyEvent is not related, that message is due to parallel make (-j 4 or similar), the error looks like it was from linking wb_command. If you could do "make VERBOSE=1" without cleaning or changing anything, it should give you the full link command that fails, which might help figure things out.

[samfux84]

Thank you for your helpful answer. I already removed the build directory after it failed, so I started again from scratch using your recommendation. I have set WORKBENCH_USE_SIMD=OFF, since I anyway set optimization flags for AVX2:

-DCMAKE_C_FLAGS_RELEASE="-O3 -DNDEBUG -ftree-vectorize -march=core-avx2 -mavx2"

This helped to get rid of the undefined reference error with the dot code.

[coalsont]

If SIMD flags get passed in by the user to the dot library, that is likely to cause it to fail to build, because it compiles some files twice with different compiler SIMD flags active, and the active #defines change what some of its functions are named.

The dot library was coded with intrinsics, to explicitly make use of SIMD well for the specific case of vector dot product (basically just used in correlation). I have no idea how well any kind of compiler auto-vectorization will work on the rest of the workbench code. I suggest benchmarking correlation (and also some operation(s) that don't invoke the dot library) with your manual non-dot avx2 build, and with a -O3 -DNDEBUG build without any explicit extra SIMD flags (on the theory that those caused the dot library to fail to build).

Another trick you might try is that the dot library is just about the only C code in workbench, so you could set the CXX flags to include explicit SIMD options, but leave them out of the C flags.

[samfux84]

Thank you for your reply. I was following your recommendation to set the options for SIMD in the CXX flags but not in the C flags. I could meanwhile successfully compile the connectome workbench. Thank you again for the very good support.

…

-- ETH Zurich, Dr. sc. ETH Samuel Fux High Performance Computing, Scientific IT Services WEC D 13, Weinbergstrasse 11, 8092 Zurich, Switzerland Phone +41 44 632 57 63, http://www.id.ethz.ch/ samuel.fux@id.ethz.ch

________________________________ Von: Tim Coalson <notifications@github.com> Gesendet: Montag, 24. Juni 2019 20:09:31 An: Washington-University/workbench Cc: Fux Samuel (ID SIS); Author Betreff: Re: [Washington-University/workbench] Building version 1.3.2 fails with undefined reference errors (#34) If SIMD flags get passed in by the user to the dot library, that is likely to cause it to fail to build, because it compiles some files twice with different compiler SIMD flags active, and the active #defines change what some of its functions are named. The dot library was coded with intrinsics, to explicitly make use of SIMD well for the specific case of vector dot product (basically just used in correlation). I have no idea how well any kind of compiler auto-vectorization will work on the rest of the workbench code. I suggest benchmarking correlation (and other operations which don't invoke the dot library) with your manual non-dot SIMD build, and with a -O3 build without any explicit extra SIMD flags (on the theory that those caused the dot library to fail to build). Another trick you might try is that the dot library is just about the only C code in workbench, so you could set the CXX flags to include explicit SIMD options, but leave them out of the C flags. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#34?email_source=notifications&email_token=AC3EHK7D4HN3E4D73GXJOGDP4EEVXA5CNFSM4H2RFJW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYNYDTA#issuecomment-505119180>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AC3EHK5JIWSF6N2QPLXLZX3P4EEVXANCNFSM4H2RFJWQ>.