Compilation fails on pacman

[aaschwanden]

Version c3508da breaks on pacman during compilation/linking:

Linking CXX executable ../bedrough_test
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)':
PISMNC3File.cc:(.text._ZN3MPI2Op4InitEPFvPKvPviRKNS_8DatatypeEEb[MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)]+0x14): undefined reference to `ompi_mpi_cxx_op_intercept'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Intracomm::Split(int, int) const':
PISMNC3File.cc:(.text._ZNK3MPI9Intracomm5SplitEii[MPI::Intracomm::Split(int, int) const]+0x34): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Graphcomm::Clone() const':
PISMNC3File.cc:(.text._ZNK3MPI9Graphcomm5CloneEv[MPI::Graphcomm::Clone() const]+0x25): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const':
PISMNC3File.cc:(.text._ZNK3MPI9Intracomm12Create_graphEiPKiS2_b[MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const]+0x2e): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Cartcomm::Sub(bool const*)':
PISMNC3File.cc:(.text._ZN3MPI8Cartcomm3SubEPKb[MPI::Cartcomm::Sub(bool const*)]+0x76): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Intracomm::Create_cart(int, int const*, bool const*, bool) const':
PISMNC3File.cc:(.text._ZNK3MPI9Intracomm11Create_cartEiPKiPKbb[MPI::Intracomm::Create_cart(int, int const*, bool const*, bool) const]+0x8f): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o):PISMNC3File.cc:(.text._ZNK3MPI8Cartcomm5CloneEv[MPI::Cartcomm::Clone() const]+0x25): more undefined references to `MPI::Comm::Comm()' follow
libpismutil.a(PISMNC3File.cc.o):(.data.rel.ro._ZTVN3MPI3WinE[vtable for MPI::Win]+0x48): undefined reference to `MPI::Win::Free()'
libpismutil.a(PISMNC3File.cc.o):(.data.rel.ro._ZTVN3MPI8DatatypeE[vtable for MPI::Datatype]+0x78): undefined reference to `MPI::Datatype::Free()'
collect2: ld returned 1 exit status
make[2]: *** [bedrough_test] Error 1
make[1]: *** [src/CMakeFiles/bedrough_test.dir/all] Error 2
make: *** [all] Error 2

[talbrecht]

I don't know if this is connected, but we have had a similar issue on the PIK-cluster for some days now:

________________________________________
In file included from /iplex/01/sys/applications/intel/impi/3.2.2.006/include64/mpi.h:1147,
                 from /home/mengel/pismDev/gh002/src/base/util/io/PISMNCFile.hh:24,
                 from /home/mengel/pismDev/gh002/src/base/util/io/PISMNC3File.hh:22,
                 from /home/mengel/pismDev/gh002/src/base/util/io/PISMNC3File.cc:19:
/iplex/01/sys/applications/intel/impi/3.2.2.006/include64/mpicxx.h:45:2: error: #error "SEEK_SET is #defined but must not be for the C++ binding of MPI. Include mpi.h before stdio.h"
/iplex/01/sys/applications/intel/impi/3.2.2.006/include64/mpicxx.h:49:2: error: #error "SEEK_CUR is #defined but must not be for the C++ binding of MPI. Include mpi.h before stdio.h"
/iplex/01/sys/applications/intel/impi/3.2.2.006/include64/mpicxx.h:54:2: error: #error "SEEK_END is #defined but must not be for the C++ binding of MPI. Include mpi.h before stdio.h"
___________________________________________

The patch to make it work for us was to switch the order of #include <string> and #include <mpi.h> in src/base/util/io/PISMNC.hh. I did not commit this modification yet since I was not sure if something else could be affected?!

[bueler]

I get the same error on my linux box as the PACMAN error if I turn off making static libraries, namely

libpismutil.a(PISMNC3File.cc.o): In function `MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const':
PISMNC3File.cc:(.text._ZNK3MPI9Intracomm12Create_graphEiPKiS2_b[MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const]+0x28): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)':
PISMNC3File.cc:(.text._ZN3MPI2Op4InitEPFvPKvPviRKNS_8DatatypeEEb[MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)]+0x1d): undefined reference to `ompi_mpi_cxx_op_intercept'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Intercomm::Merge(bool)':
PISMNC3File.cc:(.text._ZN3MPI9Intercomm5MergeEb[MPI::Intercomm::Merge(bool)]+0x33): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Cartcomm::Clone() const':
PISMNC3File.cc:(.text._ZNK3MPI8Cartcomm5CloneEv[MPI::Cartcomm::Clone() const]+0x23): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Graphcomm::Clone() const':
PISMNC3File.cc:(.text._ZNK3MPI9Graphcomm5CloneEv[MPI::Graphcomm::Clone() const]+0x23): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Intracomm::Create_cart(int, int const*, bool const*, bool) const':
PISMNC3File.cc:(.text._ZNK3MPI9Intracomm11Create_cartEiPKiPKbb[MPI::Intracomm::Create_cart(int, int const*, bool const*, bool) const]+0x87): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o): In function `MPI::Cartcomm::Sub(bool const*)':
PISMNC3File.cc:(.text._ZN3MPI8Cartcomm3SubEPKb[MPI::Cartcomm::Sub(bool const*)]+0x79): undefined reference to `MPI::Comm::Comm()'
libpismutil.a(PISMNC3File.cc.o):PISMNC3File.cc:(.text._ZNK3MPI9Intracomm6CreateERKNS_5GroupE[MPI::Intracomm::Create(MPI::Group const&) const]+0x34): more undefined references to `MPI::Comm::Comm()' follow
libpismutil.a(PISMNC3File.cc.o):(.data.rel.ro._ZTVN3MPI3WinE[vtable for MPI::Win]+0x48): undefined reference to `MPI::Win::Free()'
libpismutil.a(PISMNC3File.cc.o):(.data.rel.ro._ZTVN3MPI8DatatypeE[vtable for MPI::Datatype]+0x78): undefined reference to `MPI::Datatype::Free()'
collect2: ld returned 1 exit status
make[2]: *** [bedrough_test] Error 1
make[1]: *** [src/CMakeFiles/bedrough_test.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

As far as I can tell Torsten's patch does not work to fix this.

If I set BUILD_SHARED_LIBRARIES to ON, I get a different error,

libpismutil.so: undefined reference to `ompi_mpi_cxx_op_intercept'
libpismutil.so: undefined reference to `MPI::Datatype::Free()'
libpismutil.so: undefined reference to `MPI::Comm::Comm()'
libpismutil.so: undefined reference to `MPI::Win::Free()'

[bueler]

When I checkout a few revisions back, it works. Specifically commit 1a53a58 does
work for me. I have not run git bisect, as I don't know how, but I guess the range of possibilities is limited.

[aaschwanden]

I now get a similar error on OS X, Petsc compiled with open-mpi:

Undefined symbols:
"ompi_mpi_cxx_op_intercept", referenced from:
MPI::Op::Init(void ()(void const_, void_, int, MPI::Datatype const&), bool)in PISMNC3File.cc.o
MPI::Op::Init(void ()(void const, void_, int, MPI::Datatype const&), bool)in PISMNC4File.cc.o
MPI::Op::Init(void ()(void const, void_, int, MPI::Datatype const&), bool)in PISMNCFile.cc.o
"MPI::Win::Free()", referenced from:
vtable for MPI::Winin PISMNC3File.cc.o
vtable for MPI::Winin PISMNC4File.cc.o
vtable for MPI::Winin PISMNCFile.cc.o
"MPI::Datatype::Free()", referenced from:
vtable for MPI::Datatypein PISMNC3File.cc.o
vtable for MPI::Datatypein PISMNC4File.cc.o
vtable for MPI::Datatypein PISMNCFile.cc.o
"MPI::Comm::Comm()", referenced from:
MPI::Intracomm::Clone() constin PISMNC3File.cc.o
MPI::Graphcomm::Clone() constin PISMNC3File.cc.o
MPI::Intracomm::Create_graph(int, int const_, int const_, bool) constin PISMNC3File.cc.o
MPI::Intracomm::Create_cart(int, int const_, bool const_, bool) constin PISMNC3File.cc.o
MPI::Cartcomm::Clone() constin PISMNC3File.cc.o
MPI::Cartcomm::Sub(bool const_)in PISMNC3File.cc.o
MPI::Intracomm::Split(int, int) constin PISMNC3File.cc.o
MPI::Intracomm::Create(MPI::Group const&) constin PISMNC3File.cc.o
MPI::Intercomm::Merge(bool)in PISMNC3File.cc.o
MPI::Intracomm::Clone() constin PISMNC4File.cc.o
MPI::Graphcomm::Clone() constin PISMNC4File.cc.o
MPI::Intracomm::Create_graph(int, int const_, int const_, bool) constin PISMNC4File.cc.o
MPI::Intracomm::Create_cart(int, int const_, bool const_, bool) constin PISMNC4File.cc.o
MPI::Cartcomm::Clone() constin PISMNC4File.cc.o
MPI::Cartcomm::Sub(bool const_)in PISMNC4File.cc.o
MPI::Intercomm::Merge(bool)in PISMNC4File.cc.o
MPI::Intracomm::Split(int, int) constin PISMNC4File.cc.o
MPI::Intracomm::Create(MPI::Group const&) constin PISMNC4File.cc.o
MPI::Intracomm::Clone() constin PISMNCFile.cc.o
MPI::Intracomm::Create_graph(int, int const_, int const_, bool) constin PISMNCFile.cc.o
MPI::Graphcomm::Clone() constin PISMNCFile.cc.o
MPI::Cartcomm::Sub(bool const_)in PISMNCFile.cc.o
MPI::Intracomm::Create_cart(int, int const_, bool const_, bool) constin PISMNCFile.cc.o
MPI::Cartcomm::Clone() constin PISMNCFile.cc.o
MPI::Intercomm::Merge(bool)in PISMNCFile.cc.o
MPI::Intracomm::Split(int, int) constin PISMNCFile.cc.o
MPI::Intracomm::Create(MPI::Group const&) constin PISMNCFile.cc.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [src/libpismutil.dylib] Error 1
make[1]: *** [src/CMakeFiles/pismutil.dir/all] Error 2
make: *** [all] Error 2

However it compiles with mpich.

[bueler]

I can build both on my "linux box" and on pacman without generating any such error. I am fine with closing the issue if Constantine is.