Build Multiverso with mvapich2 (replacement of OpenMPI) in CNTK

[junjieqian]

I would like to build multiverse with mvapich2, and my CMAKE command is like cmake -DCMAKE_VERBOSE_MAKEFILE=TRUE -DMPI_CXX_INCLUDE_PATH=/usr/local/mvapich2-2.2/include -DMPI_CXX_LIBRARIES=/usr/local/mvapich2-2.2 -DMPI_LIBRARY=/usr/local/mvapich2-2.2 -.... It passes the source build, but fails the Test build with error as:
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Barrier'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Iprobe' /cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Get_count'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Isend' /cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Initialized'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Allreduce' /cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Comm_size'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Init_thread' /cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Query_thread'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Wait' /cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Recv'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Comm_rank' /cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Finalize'
/cntk/build/gpu/release/lib/libmultiverso.so: undefined reference to MPI_Testall' .

Any suggestions on this? Thank you.

[feiga]

We haven't tried with MVAPICH2. But by looking at the errors, it seems that the linker can't find the MPI dynamic library. Note that the pass of source build is because the target of Multiverso source is just a .so file, which doesn't link the MPI library. So the error occurs when you bulid test and generate the binary. I would suggest you check the path of the mpi library to make sure the program can find the path.

[feiga]

Could you try with cmake -DCMAKE_CXX_COMPILER=/path/to/mpi/bin/mpic++ .. ? I just tried and it works.

[junjieqian]

Thank you @feiga, it solves our problem.